{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SMILES enumeration, vectorization and batch generation\n",
    "\n",
    "[![Smiles Enumeration Header](https://www.wildcardconsulting.dk/wp-content/uploads/2017/03/Smiles_Enumeration_girl_600px.png)](https://www.wildcardconsulting.dk/useful-information/smiles-enumeration-as-data-augmentation-for-molecular-neural-networks/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SMILES enumeration is the process of writing out all possible SMILES forms of a molecule. It's a useful technique for data augmentation before sequence based modeling of molecules. You can read more about the background in this [blog post](https://www.wildcardconsulting.dk/useful-information/smiles-enumeration-as-data-augmentation-for-molecular-neural-networks/) or [this preprint on arxiv.org](https://arxiv.org/abs/1703.07076)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the SmilesEnumerator and instantiate the object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class SmilesEnumerator in module SmilesEnumerator:\n",
      "\n",
      "class SmilesEnumerator(__builtin__.object)\n",
      " |  SMILES Enumerator, vectorizer and devectorizer\n",
      " |  \n",
      " |  #Arguments\n",
      " |      charset: string containing the characters for the vectorization\n",
      " |        can also be generated via the .fit() method\n",
      " |      pad: Length of the vectorization\n",
      " |      leftpad: Add spaces to the left of the SMILES\n",
      " |      isomericSmiles: Generate SMILES containing information about stereogenic centers\n",
      " |      enum: Enumerate the SMILES during transform\n",
      " |      canonical: use canonical SMILES during transform (overrides enum)\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, charset=r'@C)(=cOn1S2/H[N]\\', pad=120, leftpad=True, isomericSmiles=True, enum=True, canonical=False)\n",
      " |  \n",
      " |  fit(self, smiles, extra_chars=[], extra_pad=5)\n",
      " |      Performs extraction of the charset and length of a SMILES datasets and sets self.pad and self.charset\n",
      " |      \n",
      " |      #Arguments\n",
      " |          smiles: Numpy array or Pandas series containing smiles as strings\n",
      " |          extra_chars: List of extra chars to add to the charset (e.g. \"\\\\\" when \"/\" is present)\n",
      " |          extra_pad: Extra padding to add before or after the SMILES vectorization\n",
      " |  \n",
      " |  randomize_smiles(self, smiles)\n",
      " |      Perform a randomization of a SMILES string\n",
      " |      must be RDKit sanitizable\n",
      " |  \n",
      " |  reverse_transform(self, vect)\n",
      " |      Performs a conversion of a vectorized SMILES to a smiles strings\n",
      " |      charset must be the same as used for vectorization.\n",
      " |      #Arguments\n",
      " |          vect: Numpy array of vectorized SMILES.\n",
      " |  \n",
      " |  transform(self, smiles)\n",
      " |      Perform an enumeration (randomization) and vectorization of a Numpy array of smiles strings\n",
      " |      #Arguments\n",
      " |          smiles: Numpy array or Pandas series containing smiles as strings\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors defined here:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      " |  \n",
      " |  charset\n",
      "\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "from SmilesEnumerator import SmilesEnumerator\n",
    "sme = SmilesEnumerator()\n",
    "print help(SmilesEnumerator)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A few SMILES strings will be enumerated as a demonstration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "c1([C@@]2(OC(=O)CC)CC[NH+](C)C[C@H]2CC=C)ccccc1\n",
      "c1ccc([C@]2(OC(=O)CC)[C@H](CC=C)C[NH+](C)CC2)cc1\n",
      "c1ccccc1[C@@]1(OC(CC)=O)CC[NH+](C)C[C@H]1CC=C\n",
      "O=C(CC)O[C@]1(c2ccccc2)CC[NH+](C)C[C@H]1CC=C\n",
      "C[NH+]1CC[C@](OC(=O)CC)(c2ccccc2)[C@H](CC=C)C1\n",
      "C1[C@@](c2ccccc2)(OC(CC)=O)[C@H](CC=C)C[NH+](C)C1\n",
      "[C@]1(c2ccccc2)(OC(=O)CC)CC[NH+](C)C[C@H]1CC=C\n",
      "c1([C@@]2(OC(CC)=O)CC[NH+](C)C[C@H]2CC=C)ccccc1\n",
      "[C@@H]1(CC=C)C[NH+](C)CC[C@]1(OC(=O)CC)c1ccccc1\n",
      "c1cccc([C@@]2(OC(=O)CC)CC[NH+](C)C[C@H]2CC=C)c1\n"
     ]
    }
   ],
   "source": [
    "for i in range(10):\n",
    "    print sme.randomize_smiles(\"CCC(=O)O[C@@]1(CC[NH+](C[C@H]1CC=C)C)c2ccccc2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vectorization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before vectorization SMILES must be stored as strings in an numpy array. The transform takes numpy arrays or pandas series with the SMILES as strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1,)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "smiles = np.array([\"CCC(=O)O[C@@]1(CC[NH+](C[C@H]1CC=C)C)c2ccccc2\"])\n",
    "print smiles.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fit the charset and the padding to the SMILES array, alternatively they can be specified when instantiating the object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@C)(+=cON12H[]\n",
      "50\n"
     ]
    }
   ],
   "source": [
    "sme.fit(smiles)\n",
    "print sme.charset\n",
    "print sme.pad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There have been added some extra padding to the maximum lenght observed in the smiles array. The SMILES can be transformed to one-hot encoded vectors and showed with matplotlib."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f81d8742410>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGIAAAD8CAYAAACMyXE9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAACL5JREFUeJztnV+oZWUdhp+30/xJS3JsGCZHSmgs\nvDADscIbsaTJovEiQosoELwKjIoc66agC7vpz0UUQ4lzEU5mQYMIwzgoEoT5r0wdnBkFcabR0VIy\nBBvt18VeI2fOdDzr7L322u/e+33gsPf6zt58Hzx867e+vdZ6l6qKMHneNukBhAERYUJEmBARJkSE\nCRFhQkSYEBEmjCRC0jZJT0o6LGlHV4OaRzTsylrSAnAQuBI4AjwAXFtVTyz3nbVaV+s5c6j+RuGC\ni149re3go2f00vcrvPRiVW1c6XNvH6GPS4HDVfU0gKTdwHZgWRHrOZOP6hMjdDkce/f+5bS2T733\n4l76vrvueKbN50bZNZ0LPLto+0jTFoZglBnRCknXA9cDrKef3cE0MoqIo8B5i7a3NG2nUFU7gZ0A\nZ2nDqgvS3r+fulsZZpfS125oFEbZNT0AbJV0vqS1wDXAnm6GNX8MPSOq6nVJXwP2AgvALVX1eGcj\nmzNGqhFVdRdwV0djmWuysjZh7EdNo+JcaJceSMDw482MMCEiTIgIEyZaI7rcx06CLseaGWFCRJgQ\nESb0WiMuuOjVU84NTFM9GDeZESZEhAkRYUJEmBARJkSECRFhQkSY0OuC7uCjZ8z8Im7pD5kLm9t9\nLzPChIgwISJMiAgT7K/i6Iuuzhae/p3Drb6XGWFCRJgQESbY14guLstvw6QXmpkRJkSECRFhgn2N\nGGbf3Vdd6ZLMCBMiwoQVRUi6RdJxSY8tatsgaZ+kQ83r2eMd5uzTZkbcCmxb0rYD2F9VW4H9zXYY\ngRWLdVXdJ+n9S5q3A5c373cB9wI3djiukeiqOPdZ9IetEZuq6ljz/jlgU0fjmVtGLtY1iLdZNlFA\n0vWSHpT04AleG7W7mWVYEc9L2gzQvB5f7oNVtbOqLqmqS9awbsjuZp9hF3R7gK8ANzevf+hsRB3Q\n1b69z4Vgm8PX24A/AR+UdETSdQwEXCnpEPDJZjuMQJujpmuX+Vf/CVgzTFbWJkSECRFhQkSYEBEm\nRIQJEWFCRJgQESZEhAkRYUJEmBARJkSECRFhQkSYEBEmRIQJEWGC/WX5wzBPV/qFjokIEyLCBLsa\n4XTbldWVfqEfIsKEiDDBrkZMw624J8nDnmaQiDAhIkyICBPy1K0RyFO3ZpCIMKHNzYznSbpH0hOS\nHpd0Q9OePI4OaVMjXge+WVUPS3oX8JCkfcBXGeRx3CxpB4M8jreMgchTt5ZnxRlRVceq6uHm/SvA\nAeBcBnkcu5qP7QKuHtcg54FV1YgmHOUjwP0kj6NTWouQ9E7gd8DXq+pfi//3Vnkci7M4XvjHGyMN\ndpZpJULSGgYSfl1Vv2+aW+VxLM7i2HjOQhdjnklWLNaSBPwKOFBVP1r0r1XncUz7E1XGefawzVHT\nZcCXgb9JOjmS7zAQcHuTzfEM8IXORjWHtMni+COgZf6dPI6OyMraBLszdM60qQl56taUExEmRIQJ\n9jXC6cq/NuRhT1NORJgQESZEhAn2V3G4F+euyIwwISJMiAgTIsKEiDAhIkyICBMmuo6YxqiGcZEZ\nYUJEmBARJkSECfY/+rVhGovzUjIjTIgIEyLCBLsF3SwszoYhM8KEiDAhIkyYiXXELJAZYUJEmNAm\nAmK9pD9L+msTAfH9pv18SfdLOizpN5LWjn+4s0ubGfEacEVVfRi4GNgm6WPAD4EfV9UHgJeA68Y3\nzNmnzc2MBfy72VzT/BVwBfDFpn0X8D3g56vpfNoLc+/hipIWmlt7jwP7gKeAl6vq9eYjRxjkc4Qh\naSWiqt6oqouBLcClwIfadrA4AuIErw05zNlnVUdNVfUycA/wceDdkk7u2rYAR5f5zpsREGtYN9Jg\nZ5k2ERAbgRNV9bKkdwBXMijU9wCfB3bTMgLCiS5+XOw7AmIzsEvSAoMZdHtV3SnpCWC3pB8AjzDI\n6whD0uao6VEGGU1L259mUC9CB2RlbUJEmGB/n/W4GKbQjvPsYWaECRFhQkSYMHU1YpJn9fJkxjkg\nIkyICBOmrka4n0xKpt+UExEmRIQJEWGCfbGetsv0E6445USECRFhgv1l+e41oSsyI0yICBMiwoTc\n3mtCZoQJEWFCRJgQESbYL+jmhcwIEyLChIgwISJMiAgTIsKE1iKam94fkXRns50sjg5ZzTriBuAA\ncFazfTKLY7ekXzDI4pjaCIhJr2naRkBsAT4D/LLZFoMsjjuaj+wCrh7HAOeFtrumnwDfBv7bbJ9D\nyyyOREC0o01e02eB41X10DAdJAKiHW1qxGXA5yRdBaxnUCN+SpPF0cyKZbM4QjvaJA/cBNwEIOly\n4FtV9SVJv2UMWRyTOkM36QOHUdYRNwLfkHSYQc1IFscIrOpn8Kq6F7i3eZ8sjg7JytqEnBgyITPC\nhIgwISJMsLvSbynzcuVfZoQJEWFCRJgQESbY3947LwU9M8KEiDAhIkywrxFtWKkmTMOPi5kRJkSE\nCRFhQkSYYFesx7E4cyvM/4/MCBMiwoSIMCFXcZiQGWFCRJgQESZEhAkRYUJEmBARJkSECfaXXM4L\nmREmRIQJEWGCqqq/zqQXgGeA9wAv9tbxaIw61vdV1caVPtSriDc7lR6sqkt673gI+hprdk0mRIQJ\nkxKxc0L9DkMvY51IjQink12TCb2KkLRN0pNNDuCOPvtug6RbJB2X9Niitg2S9kk61LyePY6+exMh\naQH4GfBp4ELgWkkX9tV/S24Fti1p2wHsr6qtwP5mu3P6nBGXAoer6umq+g+DnKftPfa/IlV1H/DP\nJc3bGWQWwhizC/sUcS7w7KLtZXMAzdhUVcea988Bm8bRSYr1KqjBIeZYDjP7FHEUOG/R9rTkAD4v\naTNA83p8HJ30KeIBYGuToLwWuAbY02P/w7KHQWYhdJhdeBpV1dsfcBVwEHgK+G6ffbcc323AMeAE\ngxp2HYO8wv3AIeBuYMM4+s7K2oQUaxMiwoSIMCEiTIgIEyLChIgwISJM+B+N0Vanidt+dwAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f81ddbe04d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "vect = sme.transform(smiles)\n",
    "plt.imshow(vect[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's a nice piano roll. If the vectorization is repeated, the vectorization will be different due to the enumeration, as sme.enum and sme.canonical is set to True and False, respectively (default settings)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True False\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f81d86e0710>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGIAAAD8CAYAAACMyXE9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAACLdJREFUeJztnV+oHGcZh5+fx6Sx1WKjIcQkaMFU\n6UWNEKLSm1INxiqmFyKNIgqBXAkVFZvqjYIX9cY/FyIEDT0X0lirYCiFkIaEIkhN+i+2CU1OA6GJ\naY/FBiuFmujrxUzKyTbb853Z2dl3d38PhN2Zs+d8H3l4591vdua3igjM6HnHqCdgKiwiCRaRBItI\ngkUkwSKSYBFJsIgkDCRC0lZJz0uak7SrrUlNI2q6spY0A5wEtgBngSPA9og43u93luuaWMF1jcZb\nCjfd8voV2yePXTv0MfvxGq++EhGrFnvdOwcYYzMwFxGnASTtBbYBfUWs4Do+oU8PMGQZ+/c/fcX2\nZz+wcehj9uPReOhMyesGOTStBV5csH223mcaMEhFFCFpJ7ATYAWjO0RkZxAR54D1C7bX1fuuICJ2\nA7sBrtfKTk71NjkU7f/702/Z1+UhbZBD0xFgg6QbJS0H7gL2tTOt6aNxRUTEJUnfBPYDM8CeiHiu\ntZlNGQP1iIh4BHikpblMNV5ZJ2Ho75pGQW/jLWm6o1xrgCsiDRaRBItIQvoeMY7H+ya4IpJgEUmw\niCSk7xGZj/dtnih0RSTBIpJgEUmwiCSMtFmP+lOxQWlzrq6IJFhEEiwiCZ32iJtuef2Ki7/GqR8M\nG1dEEiwiCRaRBItIQqfN+uSxaye+QfcuUmfWlP2eKyIJFpEEi0hC+k/oRkk7V5DMFY3likiCRSTB\nIpKQvkc0OU63xbjcumVaxCKSsKgISXskzUt6dsG+lZIOSDpVP94w3GlOPiUVcT+wtWffLuBgRGwA\nDtbbZgAWbdYR8ZikD/Xs3gbcVj+fBQ4D97Q4rzdpo2GOw9UiTXvE6og4Xz9/CVjd0nymloGbdVTx\nNn0TBSTtlHRU0tGLvDHocBNLUxEvS1oDUD/O93thROyOiE0RsWkZ1zQcbvJpuqDbB3wduK9+/FOT\nP9LVsTtbP7gaJW9fHwD+AnxE0llJO6gEbJF0CvhMvW0GoORd0/Y+Pxp+AtYU4ZV1EkZ60q/k2D3K\nk35d4opIgkUkwSKSYBFJSH/r1qQ2515cEUmwiCRYRBIsIgkWkQSLSIJFJCHdSb9pOcnXiysiCRaR\nBItIgkUkId1l+W18G8o4NnhXRBIsIgkWkYR0PaIJi/WESb4a3LSMRSTBIpKQrkcMY02QrR9cDVdE\nEiwiCRaRBItIQrpmPQ6N9TL+RpUJxCKSUHIz43pJhyQdl/ScpLvr/c7jaJGSHnEJ+E5EPCnpPcAT\nkg4A36DK47hP0i6qPI4lxUCMw8m4t6PTL/KIiPMR8WT9/DXgBLCWKo9jtn7ZLHBna7OaQpbUI+pw\nlI8Dj+M8jlYpFiHp3cAfgG9FxL8W/uzt8jicxVFGkQhJy6gk/DYi/ljvLsrjcBZHGYs2a0kCfgOc\niIifLvjRkvM4/I0q/Sl513Qr8DXgb5Iu/y9+n0rAg3U2xxngy8OZ4nRQksXxZ0B9fuw8jpbwyjoJ\n/iKPJTDMKwpdEUmwiCRYRBLSfTCUmSb5Uv6ypzHDIpJgEUmwiCSkb9bjdluWv3VrzLGIJFhEEpzp\nlwRXRBIsIgkWkQSLSIJFJMEikmARSbCIJKQLV2zCuJ0YvBquiCRYRBIsIgnpT/qVMI49oRdXRBIs\nIgkWkYR064hJWBM0wRWRBItIQkkExApJf5X0TB0B8aN6/42SHpc0J+l3kpYPf7qTS0lFvAHcHhEf\nAzYCWyV9EvgJ8LOI+DDwKrBjeNOcfEpuZgzg3/XmsvpfALcDX6n3zwI/BH61lMHHPYujTUpveJ+p\nb+2dBw4ALwAXIuJS/ZKzVPkcpiFFIiLivxGxEVgHbAY+WjqAIyDKWNK7poi4ABwCPgW8V9LlQ9s6\n4Fyf33EERAElERCrgIsRcUHSu4AtVI36EPAlYC+FERC9jHs/aLPHlays1wCzkmaoKujBiHhY0nFg\nr6QfA09R5XWYhpS8azpGldHUu/80Vb8wLeCVdRIsIgnpb90aFm2c5XUWxwRiEUmwiCRMbY/I9g2Q\nrogkWEQSLCIJY9cjRvlh0jDHcUUkwSKSYBFJsIgkjF2zzv6pnsMVxxyLSIJFJCF9jxi3y/Sd6Tfm\nWEQSLCIJ6W/vzd4T2sIVkQSLSIJFJMEikuD7rJPgikiCRSTBIpKQfkE3LbgikmARSSgWUd/0/pSk\nh+ttZ3G0yFIq4m7gxIJtZ3G0SGkExDrg88Cv621RZXE8VL9kFrhzGBOcFkor4ufA94D/1dvvozCL\nwxEQZZTkNX0BmI+IJ5oM4AiIMkrWEbcCX5R0B7ACuB74BXUWR10VfbM4TBklyQP3AvcCSLoN+G5E\nfFXS75nwLI4uT0AOso64B/i2pDmqnuEsjgFY0imOiDgMHK6fO4ujRbyyTkK6K/0yfTDU5diuiCRY\nRBIsIgnpekQbx+VMfaYUV0QSLCIJFpEEi0iCRSTBIpJgEUmwiCRYRBIsIgkWkQSLSIJFJMEikmAR\nSbCIJFhEEtJ9QtcGbQSwN/07TXFFJMEikmARSZjIHtGEUV/p4YpIgkUkwSKSYBFJsIgkWEQSLCIJ\nFpEERUR3g0n/AM4A7wde6WzgwRh0rh+MiFWLvahTEW8OKh2NiE2dD9yArubqQ1MSLCIJoxKxe0Tj\nNqGTuY6kR5i34kNTEjoVIWmrpOfrHMBdXY5dgqQ9kuYlPbtg30pJBySdqh9vGMbYnYmQNAP8Evgc\ncDOwXdLNXY1fyP3A1p59u4CDEbEBOFhvt06XFbEZmIuI0xHxH6qcp20djr8oEfEY8M+e3duoMgth\niNmFXYpYC7y4YLtvDmAyVkfE+fr5S8DqYQziZr0EonqLOZS3mV2KOAesX7A9LjmAL0taA1A/zg9j\nkC5FHAE21AnKy4G7gH0djt+UfVSZhdAwu7CIiOjsH3AHcBJ4AfhBl2MXzu8B4DxwkaqH7aDKKzwI\nnAIeBVYOY2yvrJPgZp0Ei0iCRSTBIpJgEUmwiCRYRBIsIgn/B51jWYeZqrxWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f81d8799150>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print sme.enumerate, sme.canonical\n",
    "vect = sme.transform(smiles)\n",
    "plt.imshow(vect[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The reverse_transform() function can be used to translate back to a SMILES string, as long as the charset is the same as was used to vectorize."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['C(C[C@@H]1C[NH+](C)CC[C@@]1(c1ccccc1)OC(CC)=O)=C']\n"
     ]
    }
   ],
   "source": [
    "print sme.reverse_transform(vect)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batch generation for Keras RNN modeling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The SmilesEnumerator class can be used together with the SmilesIterator batch generator for on the fly vectorization for RNN modeling of molecules. Below it's briefly demonstrated how this can be done."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Unnamed: 0                                      smiles_parent PC_uM_sign  \\\n",
      "0           0                      CCc1nc(N)nc(N)c1-c1ccc(Cl)cc1        NaN   \n",
      "1           1                  CCc1nc(N)nc(N)c1-c1ccc(Cl)c(Cl)c1        NaN   \n",
      "2           2                   Cc1nc(N)nc(N)c1-c1ccc(Cl)c(Cl)c1        NaN   \n",
      "3           3                CCOCc1nc(N)nc(N)c1-c1ccc(Cl)c(Cl)c1        NaN   \n",
      "4           4  Nc1nc(N)c(-c2ccc(Cl)cc2)c(COc2ccc([N+](=O)[O-]...        NaN   \n",
      "\n",
      "   PC_uM_value  \n",
      "0         3.70  \n",
      "1         1.08  \n",
      "2         1.68  \n",
      "3        12.70  \n",
      "4        85.10  \n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv(\"Example_data/Sutherland_DHFR.csv\")\n",
    "print data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAADXRJREFUeJzt3X+sZGddx/H3x64FQUNb9lpKf3Db\n0IBoYiA3TaGGEIqGtKatEUkTo4upWYmCKCayaiKJ/9gaI2LUmE2LWRMChYq20iKW0sb4Byu70Fra\nBbutC2yzbReFIv8Ala9/3FNzbe/dOffemTv3fvt+JZs5Z85zZr7PzJ3PPvOcmTOpKiRJO9/3zbsA\nSdJ0GOiS1ISBLklNGOiS1ISBLklNGOiS1ISBLklNGOiS1ISBLklN7NrKO9u9e3ctLi5u5V1K0o53\n+PDhr1XVwqR2Wxroi4uLHDp0aCvvUpJ2vCRfHtPOKRdJasJAl6QmDHRJasJAl6QmDHRJasJAl6Qm\nDHRJasJAl6QmDHRJamJLvykqdbe47/Z173Ps+itnUImeixyhS1ITBrokNWGgS1ITBrokNWGgS1IT\nBrokNWGgS1ITBrokNWGgS1ITBrokNWGgS1ITBrokNWGgS1ITBrokNTEq0JP8ZpIHknwhyYeSPD/J\nhUkOJjma5OYkp8+6WEnS2iYGepJzgV8Hlqrqx4DTgGuBG4D3VdXLga8D182yUEnSqY2dctkF/ECS\nXcALgBPAG4Fbhu0HgGumX54kaayJgV5VjwJ/DHyF5SB/EjgMfKOqnhqaHQfOnVWRkqTJxky5nAlc\nDVwIvBR4IfDmsXeQZG+SQ0kOnTx5csOFSpJObcyUy5uA/6iqk1X1XeBjwGXAGcMUDMB5wKOr7VxV\n+6tqqaqWFhYWplK0JOnZxgT6V4BLk7wgSYDLgQeBu4G3DG32ALfOpkRJ0hhj5tAPsnzw83PA/cM+\n+4H3AO9OchR4MXDTDOuUJE2wa3ITqKr3Au99xtWPAJdMvSJJ0ob4TVFJasJAl6QmDHRJasJAl6Qm\nDHRJasJAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJ\nasJAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJamLXvAuQtrPFfbfP\nuwRpNEfoktSEgS5JTTjloucUp1DUmSN0SWrCQJekJgx0SWrCQJekJgx0SWrCQJekJvzYoraV9X6s\n8Nj1V86oEmnnGTVCT3JGkluSfDHJkSSvTXJWkjuTPDRcnjnrYiVJaxs75fJ+4B+r6pXAjwNHgH3A\nXVV1MXDXsC5JmpOJgZ7kRcDrgZsAquo7VfUN4GrgwNDsAHDNrIqUJE02ZoR+IXAS+Oskn09yY5IX\nAmdX1YmhzWPA2bMqUpI02ZiDoruA1wDvrKqDSd7PM6ZXqqqS1Go7J9kL7AW44IILNlmu1I8HgjUt\nY0box4HjVXVwWL+F5YB/PMk5AMPlE6vtXFX7q2qpqpYWFhamUbMkaRUTA72qHgO+muQVw1WXAw8C\ntwF7huv2ALfOpEJJ0ihjP4f+TuCDSU4HHgF+ieX/DD6S5Drgy8BbZ1OiJGmMUYFeVfcCS6tsuny6\n5UiSNsqv/ktSEwa6JDVhoEtSEwa6JDVhoEtSEwa6JDVhoEtSEwa6JDVhoEtSE/4EnXa09Z6pUOrM\nEbokNWGgS1ITBrokNWGgS1ITBrokNWGgS1ITBrokNWGgS1ITBrokNWGgS1ITBrokNWGgS1ITBrok\nNWGgS1ITBrokNeH50DVTnq9c2jqO0CWpCQNdkpow0CWpCQNdkprwoKhG8wCntL05QpekJgx0SWrC\nQJekJgx0SWrCQJekJgx0SWrCQJekJkYHepLTknw+yceH9QuTHExyNMnNSU6fXZmSpEnWM0J/F3Bk\nxfoNwPuq6uXA14HrplmYJGl9RgV6kvOAK4Ebh/UAbwRuGZocAK6ZRYGSpHHGjtD/FPht4HvD+ouB\nb1TVU8P6ceDcKdcmSVqHiYGe5KeBJ6rq8EbuIMneJIeSHDp58uRGbkKSNMKYEfplwFVJjgEfZnmq\n5f3AGUmePrnXecCjq+1cVfuraqmqlhYWFqZQsiRpNRMDvap+p6rOq6pF4Frg01X188DdwFuGZnuA\nW2dWpSRpos18Dv09wLuTHGV5Tv2m6ZQkSdqIdZ0PvaruAe4Zlh8BLpl+SZKkjfCbopLUhIEuSU0Y\n6JLUhIEuSU0Y6JLUhIEuSU0Y6JLUhIEuSU0Y6JLUhIEuSU0Y6JLUhIEuSU0Y6JLUhIEuSU0Y6JLU\nhIEuSU0Y6JLUhIEuSU2s6yfoJM3f4r7b19X+2PVXzqgSbTeO0CWpCQNdkpow0CWpCQNdkpow0CWp\nCQNdkpow0CWpCQNdkpow0CWpCQNdkpow0CWpCQNdkpow0CWpCQNdkpow0CWpCQNdkpow0CWpCQNd\nkpow0CWpiYmBnuT8JHcneTDJA0neNVx/VpI7kzw0XJ45+3IlSWsZM0J/CvitqnoVcCnwa0leBewD\n7qqqi4G7hnVJ0pxMDPSqOlFVnxuW/xs4ApwLXA0cGJodAK6ZVZGSpMnWNYeeZBF4NXAQOLuqTgyb\nHgPOnmplkqR1GR3oSX4Q+FvgN6rqmyu3VVUBtcZ+e5McSnLo5MmTmypWkrS2UYGe5PtZDvMPVtXH\nhqsfT3LOsP0c4InV9q2q/VW1VFVLCwsL06hZkrSKXZMaJAlwE3Ckqv5kxabbgD3A9cPlrTOpUDOz\nuO/2eZcgaYomBjpwGfALwP1J7h2u+12Wg/wjSa4Dvgy8dTYlSpLGmBjoVfUvQNbYfPl0y5EkbZTf\nFJWkJgx0SWrCQJekJgx0SWrCQJekJgx0SWrCQJekJgx0SWrCQJekJgx0SWrCQJekJgx0SWrCQJek\nJgx0SWrCQJekJgx0SWpizC8WSdrB1vtTg8euv3JGlWjWHKFLUhMGuiQ1YaBLUhMGuiQ1YaBLUhMG\nuiQ1YaBLUhMGuiQ1YaBLUhMGuiQ1YaBLUhMGuiQ1YaBLUhMGuiQ1YaBLUhOeD32w3nNGg+eNVk+e\nP33ncoQuSU0Y6JLUhFMum7Dd3ppuZNpIUh+O0CWpibYjdEerkp5rNjVCT/LmJF9KcjTJvmkVJUla\nvw2P0JOcBvwF8JPAceCzSW6rqgenVdxKHUbc223OXZqXnf5a2K71b2aEfglwtKoeqarvAB8Grp5O\nWZKk9dpMoJ8LfHXF+vHhOknSHMz8oGiSvcDeYfVbSb50iua7ga/NuqY52FC/csMMKpken6udY6Z9\n2oq/0zXuY8c8V+t8jFbr18vG7LiZQH8UOH/F+nnDdf9PVe0H9o+5wSSHqmppEzVtSx371bFP0LNf\nHfsE9ms1m5ly+SxwcZILk5wOXAvctonbkyRtwoZH6FX1VJJ3AJ8ETgM+UFUPTK0ySdK6bGoOvaru\nAO6YUi0wcmpmB+rYr459gp796tgnsF/PkqqaZiGSpDnxXC6S1MRcAz3JzyV5IMn3kqx5VDfJsST3\nJ7k3yaGtrHEj1tGvHXPqhCRnJbkzyUPD5ZlrtPuf4Xm6N8m2PUg+6bFP8rwkNw/bDyZZ3Poq12dE\nn96W5OSK5+eX51HneiT5QJInknxhje1J8mdDn/8tyWu2usaNGNGvNyR5csVz9fujbriq5vYP+BHg\nFcA9wNIp2h0Dds+z1mn3i+UDyQ8DFwGnA/cBr5p37afo0x8B+4blfcANa7T71rxrHdGXiY898KvA\nXw3L1wI3z7vuKfTpbcCfz7vWdfbr9cBrgC+ssf0K4BNAgEuBg/OueUr9egPw8fXe7lxH6FV1pKpO\n9UWjHWlkv3baqROuBg4MyweAa+ZYy2aNeexX9vcW4PIk2cIa12un/T2NUlX/DPzXKZpcDfxNLfsM\ncEaSc7amuo0b0a8N2Slz6AX8U5LDwzdPO9hpp044u6pODMuPAWev0e75SQ4l+UyS7Rr6Yx77/2tT\nVU8BTwIv3pLqNmbs39PPDlMTtyQ5f5XtO81Oex2tx2uT3JfkE0l+dMwOW/HV/08BL1ll0+9V1a0j\nb+YnqurRJD8M3Jnki8P/cHMzpX5tK6fq08qVqqoka3086mXDc3UR8Okk91fVw9OuVRvyD8CHqurb\nSX6F5Xcgb5xzTVrd51h+LX0ryRXA3wMXT9pp5oFeVW+awm08Olw+keTvWH57OddAn0K/Rp06YSud\nqk9JHk9yTlWdGN7SPrHGbTz9XD2S5B7g1SzP7W4nYx77p9scT7ILeBHwn1tT3oZM7FNVraz/RpaP\ni+x02+51NA1V9c0Vy3ck+csku6vqlOeu2fZTLklemOSHnl4GfgpY9cjwDrPTTp1wG7BnWN4DPOtd\nSJIzkzxvWN4NXAbM5Pz4mzTmsV/Z37cAn67haNU2NbFPz5hbvgo4soX1zcptwC8On3a5FHhyxdTg\njpXkJU8fs0lyCctZPXlAMecjvT/D8pzXt4HHgU8O178UuGNYvojlI/b3AQ+wPKUx96PUm+3XsH4F\n8O8sj2C3db9Ynj++C3gI+BRw1nD9EnDjsPw64P7hubofuG7edZ+iP8967IE/AK4alp8PfBQ4Cvwr\ncNG8a55Cn/5weA3dB9wNvHLeNY/o04eAE8B3h9fUdcDbgbcP28PyD+08PPzNrflpue30b0S/3rHi\nufoM8Loxt+s3RSWpiW0/5SJJGsdAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJasJAl6Qm/heiYk/k\nsohJMwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f81cadfb110>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "\n",
    "#We ignore the > signs, and use random splitting for simplicity\n",
    "X_train,  X_test, y_train, y_test = train_test_split(data[\"smiles_parent\"],\n",
    "                                                     np.log(data[\"PC_uM_value\"]).values.reshape(-1,1),\n",
    "                                                     random_state=42)\n",
    "\n",
    "from sklearn.preprocessing import RobustScaler\n",
    "rbs = RobustScaler(with_centering=True, with_scaling=True, quantile_range=(5.0, 95.0), copy=True)\n",
    "y_train = rbs.fit_transform((y_train))\n",
    "y_test = rbs.transform(y_test)\n",
    "_ = plt.hist(y_train, bins=25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CBFIHcONS[r]#)(+-lon1s3254=\n",
      "75\n"
     ]
    }
   ],
   "source": [
    "import keras.backend as K\n",
    "from SmilesEnumerator import SmilesIterator\n",
    "#The SmilesEnumerator must be fit to the entire dataset, so that all chars are registered\n",
    "sme.fit(data[\"smiles_parent\"])\n",
    "sme.leftpad = True\n",
    "print sme.charset\n",
    "print sme.pad\n",
    "#The dtype is set for the K.floatx(), which is the numerical type configured for Tensorflow or Theano\n",
    "generator = SmilesIterator(X_train, y_train, sme, batch_size=200, dtype=K.floatx())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(200, 75, 27)\n",
      "(200, 1)\n"
     ]
    }
   ],
   "source": [
    "X,y = generator.next()\n",
    "print X.shape\n",
    "print y.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Build a SMILES based RNN QSAR model with Keras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, LSTM\n",
    "from keras.layers.core import Dropout\n",
    "from keras.callbacks import ReduceLROnPlateau\n",
    "from keras import regularizers\n",
    "from keras.optimizers import RMSprop, Adam"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "lstm_23 (LSTM)               (None, 64)                23552     \n",
      "_________________________________________________________________\n",
      "dense_23 (Dense)             (None, 1)                 65        \n",
      "=================================================================\n",
      "Total params: 23,617\n",
      "Trainable params: 23,617\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "input_shape = X.shape[1:]\n",
    "output_shape = 1\n",
    "\n",
    "model = Sequential()\n",
    "model.add(LSTM(64,\n",
    "               input_shape=input_shape,\n",
    "               dropout = 0.19\n",
    "               #unroll= True\n",
    "              ))\n",
    "model.add(Dense(output_shape,\n",
    "                kernel_regularizer=regularizers.l1_l2(0.005,0.01),\n",
    "                activation=\"linear\"))\n",
    "\n",
    "model.compile(loss=\"mse\", optimizer=RMSprop(lr=0.005))\n",
    "print model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the generator object for training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/25\n",
      "100/100 [==============================] - 20s 203ms/step - loss: 0.15431s - loss\n",
      "Epoch 2/25\n",
      "100/100 [==============================] - 16s 156ms/step - loss: 0.1090\n",
      "Epoch 3/25\n",
      "100/100 [==============================] - 16s 156ms/step - loss: 0.09302s - loss: - ETA: 1s - l\n",
      "Epoch 4/25\n",
      "100/100 [==============================] - 16s 159ms/step - loss: 0.0830\n",
      "Epoch 5/25\n",
      "100/100 [==============================] - 16s 157ms/step - loss: 0.0773\n",
      "Epoch 6/25\n",
      "100/100 [==============================] - 15s 155ms/step - loss: 0.07310s - loss: 0.07\n",
      "Epoch 7/25\n",
      "100/100 [==============================] - 16s 155ms/step - loss: 0.0698\n",
      "Epoch 8/25\n",
      "100/100 [==============================] - 16s 156ms/step - loss: 0.0679\n",
      "Epoch 9/25\n",
      "100/100 [==============================] - 15s 155ms/step - loss: 0.0676\n",
      "Epoch 10/25\n",
      "100/100 [==============================] - 15s 155ms/step - loss: 0.0635\n",
      "Epoch 11/25\n",
      "100/100 [==============================] - 16s 156ms/step - loss: 0.06191s - loss: 0.062 - ETA: 1s - los\n",
      "Epoch 12/25\n",
      "100/100 [==============================] - 16s 156ms/step - loss: 0.06072 - ETA: 0s - loss: 0.060\n",
      "Epoch 13/25\n",
      "100/100 [==============================] - ETA: 0s - loss: 0.059 - 16s 156ms/step - loss: 0.0589\n",
      "Epoch 14/25\n",
      "100/100 [==============================] - 16s 156ms/step - loss: 0.0576\n",
      "Epoch 15/25\n",
      "100/100 [==============================] - 16s 157ms/step - loss: 0.0562\n",
      "Epoch 16/25\n",
      "100/100 [==============================] - 16s 156ms/step - loss: 0.0545\n",
      "Epoch 17/25\n",
      "100/100 [==============================] - 16s 155ms/step - loss: 0.0533\n",
      "Epoch 18/25\n",
      "100/100 [==============================] - 16s 156ms/step - loss: 0.0516\n",
      "Epoch 19/25\n",
      "100/100 [==============================] - 16s 157ms/step - loss: 0.049911s  - E - ET\n",
      "Epoch 20/25\n",
      "100/100 [==============================] - 16s 157ms/step - loss: 0.04951s - lo\n",
      "Epoch 21/25\n",
      "100/100 [==============================] - 16s 156ms/step - loss: 0.0475\n",
      "Epoch 22/25\n",
      "100/100 [==============================] - 16s 155ms/step - loss: 0.04595s - ETA: 3s - ETA: 0s - loss: 0.046 - ETA: 0s - loss: 0.\n",
      "Epoch 23/25\n",
      "100/100 [==============================] - 16s 155ms/step - loss: 0.0444\n",
      "Epoch 24/25\n",
      "100/100 [==============================] - 16s 158ms/step - loss: 0.0433\n",
      "Epoch 25/25\n",
      "100/100 [==============================] - 16s 157ms/step - loss: 0.0421\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f6925d5ff10>"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit_generator(generator, steps_per_epoch=100, epochs=25, workers=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f691c604990>"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnXt8VOWd/9/fmUySCSgB4goJWmnX\npa1KQZDVktYqWqqumNUae9Vty2LrulZ3C8XWBaT91Qi7raWtq5Taai9KVIyx1h/Lpa1if1ZAEERl\nvdRqEqiAJBYykEnm+f0xc4YzZ86ZOZOZXCb5vl8vzcw5z5zznAn5nOd8r2KMQVEURRleBAZ6Aoqi\nKEr/o+KvKIoyDFHxVxRFGYao+CuKogxDVPwVRVGGISr+iqIowxAVf0VRlGGIir+iKMowRMVfURRl\nGFIy0BPwoqqqypxyyikDPQ1FUZSiYuvWrfuNMSdkGzdoxf+UU05hy5YtAz0NRVGUokJE/uxnnJp9\nFEVRhiEq/oqiKMMQFX9FUZRhyKC1+SuKouRCNBqlpaWFI0eODPRU+oXy8nImTJhAKBTq1edV/BVF\nGRK0tLRw3HHHccoppyAiAz2dPsUYw4EDB2hpaWHixIm9OoaafRRFGRIcOXKEsWPHDnnhBxARxo4d\nm9dTjoq/oihDhuEg/Bb5XmtBxF9EPiEiu0XkVRFZmGHcFSJiRGR6Ic6rKIqi9I68bf4iEgR+BFwI\ntACbRaTZGPOiY9xxwFeBP+Z7TkUZSjRta2X52t20tUeorgwzf/Yk6qbWDOi5+3xOOxphw1LoaIFR\nE2DWIphcX7jjDwAHDhxg1qxZAOzdu5dgMMgJJ8QTbZ999llKS0uzHuMLX/gCCxcuZNKkSX06VyiM\nw3cG8Kox5nUAEXkAuAx40THuW8DtwPwCnFNRhgRN21q5ec1OItEeAFrbI9y8ZieAf7HtpZC6nfum\n1dt5cMubPPdmR8Y55XVz2NEIj90A0Uj8fcdb8fdQ1DeAsWPHsn37dgCWLFnCyJEj+drXvpYyxhiD\nMYZAwN3o8tOf/rTP52lRCLNPDfCW7X1LYlsSETkTOMkY83gBzqcoQ4bla3cnRXZOYBObSm9gV+Aq\nzn703LhIetC0rZWZDRv56jduJrLm+riAYo4JaYbPup3bwgBPv/ZO2vZItIfla3cnz33zmp20tkcw\nHLs5NG1r9XfRG5YeE36LaCS+vR+xvsOJCx9nZsNG//PPkVdffZUPfvCDfPazn+W0005jz549zJs3\nj+nTp3PaaaexdOmx666trWX79u10d3dTWVnJwoUL+dCHPsQ555zD22+/XdB59bnDV0QCwHeBf/cx\ndp6IbBGRLfv27evrqSnKgNPWHhfBOYFNNIRWMSGwn4DAOPbBYzewufnuNIGyi+/8kkbCHE09qE8h\ntc6d61zdbhr2m0NWOlpy294H5H0Dy5GXX36Zm266iRdffJGamhoaGhrYsmULzz//POvWrePFF52G\nEujo6ODcc8/l+eef55xzzuGee+4p6JwKIf6twEm29xMS2yyOA04HficibwBnA81uTl9jzEpjzHRj\nzHTLVqYoQ5nqyjAAC0oaqZCu1J3RCNVbl6UJ1K2P7UqKb7Xsdz+wDyG1zp3rXL1uGr5vJqMm5La9\nD8j7BpYj73vf+5g+/Zjk3X///Zx55pmceeaZvPTSS67iHw6HueiiiwCYNm0ab7zxRkHnVAjx3wyc\nKiITRaQU+BTQbO00xnQYY6qMMacYY04BngHmGGO0ZKcy7Jk/exLhUNBTxMdzIOV9JNrDwc5o8n2b\nqXI/sA8hnT97El7Bgs7t4VCQ+bMn0bStlYBHiKHvm8msRRByjA2F49v7ibxvYDkyYsSI5OtXXnmF\n73//+2zcuJEdO3bwiU98wjVe3+4gDgaDdHd3F3ROeYu/MaYbuB5YC7wENBpjdonIUhGZk+/xFWUo\nUze1htsuP4O3xf1Jt82Mzfj5Zd31dBpHFIlPIa2bWsNnzz7ZVeg/e/bJ1FSGEaCmMsxtl58BwM1r\ndtJjTNqxrJuDE1e7+uR6uHQFjDoJkPjPS1f0q7PX60aV69NQb3j33Xc57rjjOP7449mzZw9r167t\n83O6UZDyDsaY3wC/cWxz/ddnjPlYIc6pKEOFuqk1EPxOagQMEKGMZd3pglgZDnG4q5uLzFMsKGkk\nTBfdJkBQYsiok3IKm/x23RlMf8+YlMid895/Ar99eR9t7REqSoO0tUe4cfV2z2MERbjt8jPSon0y\nRzLVD2hkz/zZk1LmBt43sEJz5pln8sEPfpD3v//9vOc972HmzJl9fk43xLjcxQcD06dPN9rMRRnq\n2EMmrxn5LAtCq6mI7IVRE9j8vn/l6s3vSROoK6bVcHjL/fyf4I9T/ATdwXJKLvtBiqjmGpLpFGw/\nCPCnhkvSts9s2EirixmlpjLM0wvP9318v7z00kt84AMf8D1+IPMrCoXbNYvIVmNM1kRaLeymKAOE\nU2h/dmgGP2MGoytCLP7YaXGT0EnpArV87W5WB1anOYhLeo7AE19Pxvx3hsex6fAVtHZ9GPCXQ+Dm\nCM2Gl6mkv+3quVI3taboxL6QqPgrygDhJbQHO6MpIu0UqJtWb6e6zN1BbCLvIJF3AKiI7GGZ/JDl\nZT+k1VSxrLue5mgty9fu9hS9XIVZgPPe7+6vqK4Mu678+8OurmRHC7spygCRSWgzhR2OCoc8o3yc\nztuAgAhMCOynIbSKOYFNtLZHkvHsTodsZUVuteEN8PDWVtf4eCuSyU5/2dWV7OjKX1EGACtk0oqc\nmRPYxIKSRqplP23WKr29lqZtrWz58zv88pk3sXvnlgXqaQitSjH9GBMXei8qpIsFJY00d9Vy85qd\ndG17gJl/vpOn2E9baRXL3q2nOVab9jnXudnGWTcq59OE9b7Y7epDFRV/RekDbmnayf1/fIseYwiK\n8Om/P4lv152R3GcXcyu71xLyCRJfpROFG1cfO6ZThB/s+SizAtupkf0Y4qv8bFTLAeYENrFY7mPM\nnw8lbxb2czbHahHiq/orQn/gWwH3udlvAF5PMcPdrj6YUbOPMqTor3otmbilaSe/eObN5Kq+xxh+\n8cyb3NIULx/gXMW7Zfdaq3SLW0vu4Y7QncnyDxMC+7ky+CTLuutpNVW+hB/goBlBQ2gVYwOH0p4S\n7Oc0xKNy5gfTHcvOuQE5m4uUgUdX/sqQoSAVMgvA/X98y3P7r5/fgzO42iu7tyaxfU5gE58Prk8T\neEuEvT7vNAN1mlJESC8jkTKXYxnFbe0R/qZsX7ojwTHOOtdwpxAlnQHuueceLr74YsaNG9dncwVd\n+StDiP6u1+KFWwastb09Ek3b7uW8NRwz9Xit7KvlgOfn3zEjaYlVETNCS6yKhdG5VHIo49ztGcXV\nlWHfmccdLtc13LBKOm/fvp0vf/nL3HTTTcn3foUf4uK/d+/ePpxpHBV/ZcgwWOLKgzm211vWXU/M\n5X4REDKu7CEuwm4lHjpNKbd2X01t1wree/SX1HatoDlW610LKPEZK6PYisp568z5RFyO7cw8Lsrw\nzR2N8L3TYUll/KePMti95d5772XGjBlMmTKF6667jlgsRnd3N5///Oc544wzOP3001mxYgWrV69m\n+/btXHXVVUyZMoWuLu+ntHxR8VeGDANZr8XO2e8dndN4twgbi0wr+5ghGXmzMDo3ZZX/YM9HWVDS\nyOtln2FT6Q3MCWwC3GsBGRN/SlgYnZt09l4xLe6oPWvOtbww7dvs5QRiRmijim/2/HPKnIsyfNNq\nKNOLPgi58sILL/DII4/whz/8IVmr/4EHHmDr1q3s37+fnTt38sILL3D11VcnRd+6CeTyxJArKv7K\nkGEwxJU3bWvluTc70rbPfN8YajLchLwE3mtlHzPw854LkiLcHKtNrvKXdddzZfDJFOfwHaE7ubXk\nHtcbxVej13Hm0ZXJYxlg9bNvJZ3lZ825lnFLXiVwazvVS17j3Cv+Ja3oW9FF9PRjQ5n169ezefNm\npk+fzpQpU/j973/Pa6+9xt/+7d+ye/dubrjhBtauXcuoUaMKfu5MqMNXGTIMhrhyr6zdNw5EXIuJ\nWSzrTo/bt8wrzbFaiFomoAPsYSy3R+Nml02lNyRDP/8rdhUjpn+ar2xLjx4KCHw+uJ6tsb9jffCj\n/Dpa62pqsojGjGcm8JAI3+zHhjLGGL74xS/yrW99K23fjh07eOKJJ/jRj37Eww8/zMqVKwt+fi9U\n/JUhxUALUya/g/PmZNdep8BbK377yr65q5ZQUMDARTyVlhtwe3AV33khxHjcfQQBgYZRj1Dx9dto\n2tbKkuZdrg7obNcyJBg1IWHycdleYC644AI++clP8tWvfpWqqioOHDjA4cOHCYfDlJeXc+WVV3Lq\nqacyd+5cAI477jj++te/FnweTlT8FYXeVb90G5+tno395uSsemkJPByL8rlD7kzJqu2OGYyBBaXp\nq/tSc5S5Xb+gjSomeDiJKyJ7YEcjdVPrqZta41l50z7nIcmsRWkltPuqocwZZ5zB4sWLueCCC4jF\nYoRCIe666y6CwSBf+tKXMMYgItx+++0AfOELX2Du3LmEw+GcQkRzRUs6K8MetzLG4VDQ05adaTzg\n+1he5ZOdGb8QNwFZDlmA18s+4xr+GTPCjdGvcEfoTu/Er1A42TylaVsr8x96nmhPqg6EAsKvzn6L\ns177QdwUMmpCTn0CBoJcSzqzozFZAbUYrs8NLemsKL3AWr27rXy96tVA5nwCq06961OEQ2zqZi2C\ny2cmx1aUBjnc1ZMx49d6Mmgz7qv7NjOW5lgt03r+1zUxDDjm2Jxcn7y+Wx/blWwPWRkO8eOpf+Ks\nnYuPrYytaBgoOoH0ZPLANpQZaFT8lWFJtqYlcwKbWNDZCEsOpK0Ks+UTuPodrNBCm5h2PvwvbA9d\nx/xL5iXHN21rpfrR1OxZC3tWbSYHMcDi7i+yNfZ3fL/0Tvc+vR1vxWPbZy1KmoBS+N7p3tEww1gw\nhxIa6qkMSzI1LbHMLhMC+3GLAc81n6BpWyt713wjTUwrpItboivY9MidybDKuqk1BDycjvasWnvI\npjHQbQKU08V3Q3fxp0Rsf2lJgEh4vPeXkCm2vR+jYQrJYDVj9wX5XquKvzIsyRTJ4mZ2sceA55JP\nYD1h/I3Z53quEomxVFbyx0fvYsm3F9Oy6H2YjrfS6v+4ZdU2x2pZ1l1PF0FKJEZA4sez6vcvlZU0\nRyanN3j3uK4UvKJe+iAaplCUl5dz4MCBYXEDMMZw4MABysvLe32Mgph9ROQTwPeBILDKGNPg2P9v\nwFygG9gHfNEY8+dCnFtReoNXVA5AdcDd7GKtenPJJ7CeMNpKM0TgSBdfi91DONpFRcBWnz/xX1ss\nvYa+xeKS+ygT9yeYCumiNraVhd1zWVDSSI3sd6/377aa78domEIxYcIEWlpa2LfP/UY71CgvL2fC\nhN7fjPMWfxEJAj8CLgRagM0i0myMedE2bBsw3RjTKSJfAZYBV+V7bkXpLW4JV1ZUTuB32WPA/eYT\nWE8YbjZ6O2MkvcSyAJHweM7r+C5Rj4ysMZK5UFu1HEiGkG4qvcH9BuS2mrfs+kUUDRMKhZg4ceJA\nT6NoKITZZwbwqjHmdWNMF/AAcJl9gDHmt8aYzsTbZ4DB++yoDAvqptZw2+VnuJcpmLUovsq108tV\nr+UHsGz03Sa3P7lwZA+/OvutjKUhMmH3E7iVich4XZPr4aYXYEl7/OcgFn4ldwph9qkB7MukFuDv\nM4z/EvBEAc6rKDnhlphlhWamUMBVr/0Jw8ridYvSOUIpY1zKLQswdds32VhSQajs3bTM34OMdP2c\ndVy7n8A6/9dD8SziSHgcy6JXce+vRlD9m43aYnGY0a+hniLyOWA6cK7H/nnAPICTTz65H2emDHVy\nbvTSixjwTFnCVj7BYwkBXlxyX9JkU0aUMF3EjHsrxhITpSTaAZLeRnFJ9Gr+M7SSUulOjjcGYsCW\n2KlpWcJbj7+QzbOvB/D9feSa/awUB3ln+IrIOcASY8zsxPubAYwxtznGXQD8ADjXGPN2tuNqhq9S\nSLzKGNRUho+t/vPI+HTLGwgFhRGlJXREolRXhjl8tJv2SNQ1g9ciWxN2+7jWhKADLAndx2hS/QbO\nY0UoI3z5D2Fyvb/vw+O6MmU/KwOP3wzfQtj8NwOnishEESkFPgU0OyYzFbgbmONH+BWl0GRt9JJn\nffftj69knfxLsn7+rSX38Nvgv/Jc7EqeKr2Bae+uSxZRWxK6z9Px67cPjBXO2RBaBUCnKU93GDve\nhzkKG5bStK3VM9LJ+T15ZTPfuHr7gPVIVgpD3mYfY0y3iFwPrCUe6nmPMWaXiCwFthhjmoHlwEjg\nQYn/i3zTGDMn33Mril+yFVzLWN892+p/RyMLoncmwzQnyH6ulvVJ8bWbagBGZ2ml6Hf1D/Fwzu+H\n7vQ3GDAdLUnzjhvORuyZ8iEGqkeyUhgKYvM3xvwG+I1j2yLb6wsKcR5F6S1eoZ3JxKx8Mlo3LE1b\nyTvF26rN47bPyUFG0hkrp1oO0M4IRhDxjOX3czw7f6HKM7MZ0huxZ8qHgMw1kJTBjdb2UYYFGROz\ndjSCBMC4iKKfjFafJQ/itXky+9g6TSlLolenJHRZ5Z09k7Q8SHuCCIW57fCVGT/jbMSeqQGNxZCu\n+z+EUfFXhg0ZC665Cb/f2H6vxiAO9jCWitIgo6N/SdtnOXD/s6ee86+8nidtjVasJK1MjuLkcQBB\nIDwa6TkKXYfjO8Jj4KLb2fKbKsgg1s76RM5oJT+fUYoDre2jDG/cbP0AEkzWvM+KW1KYgwhltE1b\nwOhLv013sNyxr5Qbo9dxacl/87vS87hp9XYgHi1kx1nMzY09VMHlK6E7ckz4If4e97pEFl71ieqm\n1vD0wvO546opA94jWSkcuvJXhhw5xaV7mWxMzH+cv1tS2Kkfh1f+J/k+PGsRZyXGlTjGhmct4rye\nmfyPzbzSHokSCggVoQCd0VjyVM2xWtaZc7mw+/euyWLreqZwzSNfTn+SSTiv6256ATi2kg+K0GMM\nNT7i9wdDj2SlcKj4K0OKnJO5CtXLNVtS2I7GeI18jxyC5Q0b0+zq0Zjhb44v5zuzJ6UJ7o2re9J6\n/m6ITeHK4JPuJiwg1tHC+xY+npdoD3SPZKVwqPgrQ4pMXbZcRStD9cqCZba6NHJxdsXK1vjded5b\nH9tFc+exnr8Am0pvyOgPaIuNxaAhmkoctfkrQ4qsyVxOJtfHbfujTgIk/vPSFTT1zGT+g8/T2h5J\nCub8B5/vXVJTphyCBKPCIdzwcqYuvvS0NJ9AtUfJaEiv82PdEJXhi678lSFF1mQuN5wmmx2NzHj0\no+wO7U/WxGmO1RKNGZY07/K1WrY/NbxW3uK6yjIdLdR6lFmAeBN1L2eqm/39iIynIrInbWy3CaQ0\nf7dobY8w0WYGch5P7flDGxV/ZUiRNZnLBbtQXzPyWW4xd1HNEddCau2OOHiv49nn0BYbm2gJmcpB\nMyJN+K2Y/mrZz16pojp4G+DuS0gzB/36YthyDym5BKEw3zbX0nx0husx7E81CER74p9V09DQJ+/C\nbn2FFnZTeks2W719f2VFiENHupPNUjaV3uAq1C2xKmq7ViTfC8ckdnRFiMWXnpY8h7No2pzAJpaH\n7k7L0u0yJXwtOi+5IneL4+8yJfzVlDNaDrGHKm6P1vNU+XkYE48IsqJ1/ilx0yrpOWI7g8D0L9JU\n8+9ZE7W8cBZ6UwY/fgu7qfgrwwq3KpV2Xi/7jGtZ5ZgR3nv0lxmPbb8hQOoq3iAEJf1vzX5T8brx\npM4Dft5zAYu7v5iyPdNN66qKH3Pe+0/gty/voy3hw/CLAH9quCSHTygDjV/xV7OPUjT4ib7JNOaW\npp384pk3M56jzbj32hUMr5V9lkBCOg8yMq0MgyWqcwKbXEosu0tuvOSD9Tqz8EO83v/ng+vZGvu7\nlHN7fbZaDtDaHmH1s2+x/MoPUTe1xrOcs+vne5G9q/X/iwON9lGKAmvFbo++uXnNzpToG68xtzTt\n5AP/8URW4QePVofEa+QExSASfz1GDvH90J38KVHCeU5gE3DMdOPWk9cNq83inMAmYj7/HANCskjc\nseNUZTx+NGa4qXE7TdtaXbN8QwFJix7qTfaun9+TMjhQs48yYOSyQvTTfCSXFW0melNIrdOUsjA6\nlwUljVlNNxZHTZAoIUYQt9PnUrTNaYZy8xdYc7I/IViNWCA9ssdtW7YVu/N32NnVzcHOdKe4+g76\nDzX7KIOaXDNx/cTvZ6ouabe/28M33bAKqb1e9hn86rFVsjmb6cZaax2mjBEczViqORP2xuwWEVNK\nmLj4u5ml4Fh8/9MX76eubCmUt0DZBAjGM45zMc+4/Q4956uVPwcdavZRBoRMmbhueNmeKytCzGzY\nyMSFjxPwWDpbq+IJgf0EEh2w/jO0kq2l85KdtyyzjR0vU4oX8TILmT8jEhfmCo7mtNK340zYsq5v\nbOBQ0ixVjnem7/R31+XVtczC7XfohVb+HHyo+CsDgtdK0Gv16FWN8mBnNGlf7vEwYS4oaUwre1Aq\n3YwNHEreDBpCq1Ls9ptKb6Ba9hPLwSrazgjCHPGsuGkxmkOuEUXZMCYeveM05bhdn715jJObSx/M\nmnHsB7+rea38OThRs48yIHhl4gpxc4JlfrDblEeFQ5SHAhzsjKaFVWaixkcUTVIsu0mznftpq9hl\nSuIdtwK9M+Nkw81+b5Ep0sdJOBTkRDy+D59NaZLH9/gdVoZDjCgr0WifQY6KvzIgzJ89iZtWb08T\ncAP8e+Pz3LR6O5UVITo6o1gFja0yx6MrQq5ORTvhUDBpkughQAmxjOMhLpZuq2gv4bdW+K2mijBH\nGBvI3JsX4iIeMaWMlexj7efxEn7wDk+1/AI1leEUIZbfFaaSqVc29ZI5p6nY94L+DpFVs48yINRN\nrfFcufcYgyFu0nFK9kU8xWPdX8loq6+pDHPb5WdQUxlGgIBkF36Ii6WfWHuLd8xIJh79FbVdKxjt\nQ8wtEf917OyspiE7rabKU/jBPTzV8gtYUTZ/ariEpxeeHxcTt+YzfruW2aibWpPyPVvfuwp/7gxE\niGxBxF9EPiEiu0XkVRFZ6LK/TERWJ/b/UUROKcR5lSLBqmW/pDL+M+FYrMnRCejmuP1+6E6eK5uX\nvAmEQ0HOe/8J3PrYruQf0l6yO24tsfTr5I2ZeKy/dQPy+7nmWC2zAttzCiG1O3fd+LWp5aejb6TV\nVBEzkvQLrAue625r96hk6rt5jQ2ry1fKzUXJmVwDIApB3mYfEQkCPwIuBFqAzSLSbIx50TbsS8BB\nY8zfisingNuBq/I9t1IEZKhlP3/2zJxqzniZZMZwiIbQKkZICcEp9aze/FayQBlAQ1d9mh3fGWMf\nSaycl3Wnj3ViDEmH7QTZz/LQ3UQpyeobsGbk9XRhDBwyZUQlRCWHaTNjM4akWsQM/PDANPZMm5Ms\n4VBdGea2jBnQI6iuXMH8y9QePxjIuRR5ASjEyn8G8Kox5nVjTBfwAHCZY8xlwL2J1w8Bs0R6G+im\nFBUZatk7zQbBLP8kMplkKqSL20Y9wuM79qQIv0XElGJMXGDfMSN5oOc8AsSSoZFjA/EbCMRNM5nM\nMs5plkkPI+VY6KbXZ62nA6+nBJF4FFLiKN4TcCES7eG3L+/LuAp3mhamvbuOs5o+inE8kSn9j1co\nbF+GyBZC/GsAu/eoJbHNdYwxphvoANKyVERknohsEZEt+/btK8DUlP6iaVtrMt5+ZsPGY7ZKrwiS\nxHa72eC/6j/k2Vwcssfdm46WNEewVwz8pcFnPMMjm2O1tOYY429HhLQQUbv5xquEBMRvJGPEPQQ1\nG9lWiXbTgvW91Mh+JI9Yf6UwuIUy93WI7KBy+BpjVhpjphtjpp9wwgkDPR3FJxmdVV4RJC7bsz0J\nZBJNgL+42Pa9YuBH4+6grZb9bCq9gZocY/ydCPGYfLsN3jLfNMdqsz5d2Oe6oKTRl3+kssK9G5h1\nY7aHZbp9L72J9VcKw0A4zwsR6tkKnGR7PyGxzW1Mi4iUAKOA9CBkpSjJ2Df3Yu8euW7YG5Q4ywc0\nx2ohCreW3kclh1JLL4TC3Hb4yrTj5RK9A3HRtmrzCP5i/N1oNan1/500x2pZYBpdQzSdVMsBX7bf\nQ0e6U3IkwLuEtef3kmOsv1I43Ho19yWFWPlvBk4VkYkiUgp8Cmh2jGkGrkm8/iSw0QzWinJKzmR0\nVuUaWWKLDKr73WzuO+vPjLataJtjtUw9spL5sevpDI9POeaW4y9Mn4OH+eYdM9J1Ze8U+t4Iv58I\nHYg/yfj5K4ghXOrD9BONmbToEK8SDJ4mtBxj/ZXiJW/xT9jwrwfWAi8BjcaYXSKyVETmJIb9BBgr\nIq8C/wakhYMqxYPTvu9lbhgVTtTd+dUIZh5dQdNlu+CmF9yFf0cj3D4R1vxzSs2Z05/7Dz4Reypt\n+ENdH+aM9u8y8cgv48fumelqN/WKgb+1+2rfRdv8YAyuJp5MNMdqeceMzDquRGK+bf/OG7HXjXlZ\ndz0RylK2dZpSlhy+QssvDxO0pLOSE25mhFBAUvq/em2zygmnPdo6w0EdOFsoumEvVXzj6u0p+45V\n9DyQEj7pp3OWX/zM0Q23Usz5nMNZOjljKeyL99P5xCLKO/emfC+evyelKNA2jkqf4CUmznouXnXd\nR1eE2Lbo46kbv3e6e7mBBH5aKMIx4Zu69H+yln8AuCywieVlP6HUHPUc47T5HzVBBLGFZGauu2Ph\nLCn9fT7Fg10fTtmXrX9Atu8hFBBGlpfQ3hlNqdHvVoLBEnc/fRKU4sKv+A+qaB9l8ONlRuiIRFNi\nzNs9xPdgZzTdrJDFyehWu96N1vYIMxs2+hJ+gEdjtXwn+BU6w+M9be+HKUuJ2pkfvZavRed5RvK4\n4ZaZvLzsJ7zxmcO80XAJ5195PatKP0dPlj/HNjMWAWa+b0wy+seKiKoMh0Di36894grIGEUyEMlF\nyuBAC7spOeFVydGZjFKZofja8rW7U00KozwKjeHuPA2KuJZvFjI3FIH0FfjyznoqvvMym5vv5rSt\nt6SYX2IGHu75SFqzdIDmrlo1hHN1AAAgAElEQVTPeTjPUSFHvMMqJ9dTF3yaOrkbMtQgsr4HAzz3\nZkeaWWZmw0baI6nfdyTaw5LmXWxf/HFPE47f36cy9NCVv5ITfpJRmra1cuhIt/OjSdJWlbMWpTkf\nrUxc56pawDMZLJsB020F3lD6E9jRyFlzruWhnnNTIoACAlcGn3R1tIaCwnHl6Wsnt3N45RQkn3jc\nsqCxnMjpNfzdar54rdTbIy5PWqTG/jstTVp/f3igK38lJ6wVpFv/15kNG2lrjxDwWBFbpK0qJ9fz\nwhsHqd66jPEcyFjTproynJzDkuZdaatdJ/ZVeIwAJY7VdZijyRX4peEdBByHS2b9dh2by4jSIF3d\nMddz51ISOhlW6WH2Mnjb+J1i77WCh/QnLafT3kCyP0KN1t8fNqj4K9nZ0RgXyI4WGDWBulmLqFt4\nzBTjFJNMwh8Kiuuq8qw519J00j+kNG4JdXWnRQtZn62bWsM31uzIOG1nJE3Ao6a/6XgLs3gUlR4i\n7WyKUllR6im0fpPKuoPllFiJbh5mr0y+DucNdP7sSWlRTsnjOObqFvtvCb86eYcPavZRMmOFYWbo\n9+oUE6sNolvN/WiP4cbV21Pr/ySw1/nZvvjjLP/khzwdlU3bWumMZq7T71rCwIV4zX884/7tIjy6\nIpTRGZq1/lDCjPNt+fKxfIdZi+gOlqeMy5Qo5maWqZtak5IMZ8d5o1AnrwIq/ko2MlTltLCLhqtd\n3SVByU+ziky14v3UOc+1tIMbThG2omncqAyHWFX6uYz1hwxCbdcK7j0049jGyfV8W76cMYIoKJK1\n5sviS0/zVRxsICpIKoMPNfsomclSlRNS7c2Zmonb7eZgq//j075sb3O3pOQePlu2kSAxegjwy57z\n06JyDpqRObVLtLBbrSIZhNyO1b4QTmPRI900cGeafwGOPUU4hfbeQzP4GTPSxlvEjOFPDZdknIOX\nP8b5/Xq1X1Qn7/BCxV9JwdlHdF14HBWRPekDbTVg7P14c2kmDomnBrtPITwaeo5C12EAjoZGcZv5\nAj87NCPplLy15B6uDq5POlJLiHF1cD1Ayg2gtx0jDMeatYyVRJ3/KJ6x/HYn6cyGjbR2fZiuQCwt\nc9d6iggF0v0emUJjrf2WQz1Tf1c/xcH83iSUoY2Kv5LE6bhtbY+wqPQKGkKrKOk5cmygoypn3dSa\npLMxWzNxJ9eMfBYeu/uYaSnyTsr+smgHN5sf8E7gWgDPTFgR+FxwfYr4V3qFWGYgZuvSZeH15AJx\nP4HdSWqZwKwKpG5lJcBw62O7gPh3ly00NhQUDh05ljFtT+CC3ol4f1eQVAYfKv5KErcokIe6PszI\n0hKWjHo4Ge3DrEVpxdlqEqYftzaIEcpY1l2fXLlbhENBFoRWQySzo7FMelgSuo8RRCgT75aPAeI+\nB2uF7nUjctJtAgQwtJmx1OT45OI031wz8lnmdv0imeDlFbJ6sDOaFPDla3cT9WgeUFMZ5vDRbtcE\nrj8+ehfXm1/xFPtpK61i2bv13Lwm/r2rsCvZUIevksQr2uPeQzPi1TiXtLtW5Wza1kpnV3zlajUq\nsZyXneHxhC//ISu+cxvfu2pKSvTOFdNqKI/s9TW30RzKKPwQX/0vCd2XfL+su54uk3l9Ywy8SwU3\nRr9CbdcKzw5eXk8u573f1nRoRyO3mLsyNpi3Y/k8vL5366miwyWfYE5gE/9h7qJGUh3rF/b8vk+b\nfitDBxV/JUlvokAsU5HdXt0cq+Ufgv9Nc90uKr7+Mk09M5nZsJGbEqah7101hfmzJ/Hw1lbaYv7q\n9vhlNIeSQtscq+Vr0Xm8Y0Ym+/ceMmX8NVaWdOqKwBg5lBTp182JGVswOvnty7Z2oxuWpprHbMf3\nKslsmWvcsLa77c/kWPcVsmnrm1DM/Xs924cqWVHxV5L0po+oV7OQEWUlSXu2W4vHJc27iER72BCb\nkrWhyVHj3dfXiXP13xyr5cyjK5l49FdMPPorTj/6Uzo4ztVnMEYO8ZHArhSbf8zAgz0f9XT2trZH\nksJjMhSos4TZiWWnz/S9u+33NEPJgewhmz5yN4qBjO1Dlayo+CtJetNH1GuVaVXYvHH1dtcWj5YN\ne1Zgu2tUjrVS/2usjMPkFn8+mkOeSWaQOf7fOZeAxOeYCUt4WrM8xTgF2xL4bN+72/4jFeNcz7GH\nsdlDNn3kbhQDmdqHKtnRev5KXnjVg3c6d714vewzadE1kBD/xGu3/dYYrxuHfbv9n/g7ZmRyle8X\nv/0EsjVmORg6kX8I3lWY8EqXBjgRynjhzG9x1pxrM392SSXuvx2J+3WKhIkLH/e6iqw5EUMZv/X8\nNdpHyQu3hCG/wg/eETmSodyCfYyf7fb3Y+UQUSN0mZKUhiyZ6HRUHPXCCu9cXHIfY+RQynk7TSnL\nolfx9Dddauc4aiclw2id2+yOduu1bUx41iLO8uqNbMerhHaR9e/VctT5oWYfJS/qptZwxbSaZFOR\noIhv4Qf3Hrt9TUgMfzXlHIiN9NVAvYIjfLr8GYJejyA2mmO1TOtayVej16WVa7j/yNnpH3Czvzdd\nB4/+S3ab/OT6jFFYnsxaFM/VsOPI3SgGeuOjUo6R18pfRMYAq4FTgDeAemPMQceYKcB/A8cDPcD/\nMcaszue8yuChaVsrD29tTVby7DEmp5W/MxlKMDln5nqZfzIxJnCI2vJHmP7uOm4ufZAT2Y/XbSsg\nsCC0moe6ziFzsOkxmmO1rklhabjZ32Mumb625i954/LU4Ja7MdjRTOX8yMvmLyLLgHeMMQ0ishAY\nbYz5umPM3wHGGPOKiFQDW4EPGGMyGhfV5l8c2G3+zg5WXglOmehNU/Vuk16nPysSgMUHU00uEgDj\nLu9edn+rm5efG15lOMT2xY7+xZ72d9dJF5VNXhkY+quH72XAvYnX9wJ1zgHGmP81xrySeN0GvA2c\n4BynFCdtNuF3VvO8vdQ9tj0TbmagWCLy50BsZFrSVqcp5Zc953t+xnNtY2LpJhcP4QfvJK+YMbzR\ncElKAtvoilAy29iKOnq67AZ+PPVP6QfIxc5eZDZ5ZXCTr8P3RGOMVfVrL3BipsEiMgMoBV7L87zK\nIMFyurklHYXp4ruhuzIWRXOSqSaOAJcmny7i+zbEpjArsJ1yupJC3yMBNo+9jKv31vO/oU97nyxD\n+8Q0Z61HkpflXHTWytncfDenP/eTeKcwoIb91OxcDKeMTjWvzFqUFrVDIBSfQI/t+yxCm7wyuMkq\n/iKyHnALKv6m/Y0xxoiI5/OriIwHfg5cY4xxfUYXkXnAPICTTz4529SUQYAV7eMVO18isaxVMZ14\n2csN8Lj5SHKfV2hlCTE+dOBxLmI873iVdQ6P8W6fKNAaq3IpyOY4RAbn4lmv/QASwp/EzW7vZX93\n21ZkNnllcJOvzX838DFjzJ6EuP/OGJP21yAixwO/A75jjHnIz7HV5l88NG1r5exHz2Uc+zzHtMSq\nqO1aARyzk/eWcChIJNqT1T/QEov7HZaH7k6tCxQIQd2dCXFND3nsDI9n2qE7XDOXLbL2uh0isfRK\n8dFfNv9m4JrE62uAR10mUgo8AtznV/iV4qJuag3jLv9OevigDSu7NRwK8um/PyktRM8vVvZrTWU4\na6euGtnPgpJGHug5LyXskro7aeqZyZLDV6SHmYbCVFy0lNsuP8O1LWI4FOSOq6akdRZLw8s+r3Z7\nZZCQr/g3ABeKyCvABYn3iMh0EVmVGFMPfBT4JxHZnvhvSp7nVQYbk+vh0hV0e/yTajNjk5U87//j\nWxlX1QAVIffjtLZHWL52N/NnTyJQeVLGY0jC8Xxl8EmWddfz3qO/5CPRFTT1zOTmNTv52aEZaRVI\nuXQFTK6nbmoN2xZ9nDsclUizlbtIMkRi6ZWhi5Z3UArK5ua7OX3rLYQdHay+JV/m/iNnZw2JDAWE\n5Vd+iOVrd7tmb1qEQ0HuO+vPnLVzsavT1ond7FTjkRlaUxlOacziC7fsXMs2n2mfDWf3NI1VV/JB\nyzsoA8JZc65lM3DSc8v5G7OfvTKW5T1X8Uh3PLs121JjZHm8GqhV/tmLSLSHT/2/k3jgnFs567Uf\nYDreoocAQRNzTfiyzE41lWHPYnS+SiHbcdbX6XiLyJrrWfjANrYcfyHzZ8+k7qYXMh7CrXua1eRF\nbwBKX6LlHZSCc9acaxm35FUCt7ZzZfmPeaR7pu/PHuyM0rStlUoXe7uTHmO4evN7uGXi/dSWP8Kp\nR37BX8Q9haTNjE1G5/Smb4EbnU8sSnvqCHOU+SWNvssLa2VKZaBQ8Vf6lJxX08DNa3ZyNItPwCIS\n7eGXz7yZrOm+tvtDns1YyhN+hELUhGna1kp5p3sXMuspw4+IF+wpRFFyRMVf6VN6U2ExEu2hM+q/\nXIOl9XMCm7gy+KRnMxZ739xc+xY4Wb52t2fWr317NhEv1FOIouSKir+SO24tAD3aAs6fPSlraeZC\n4ZZl7GzGYq3G66bW8PTC8/lTwyXZwzZdaEs0q3eGijqzgbOJuNtTSCggdHZ1a2tCpU9Rh69yDD/R\nKb/+N9hyD8n1tlWC2F6OoOMtImv+ha8/sM01MzZTAbh8isN5xf07O2jZV+O9jbSprgzT3O5digJS\nRdzr2M7KlKPCIQ53dSd7IqsDWOkrNNRTiePSGYpQOBn3nhyzZh5+q1Dawyst3EoyGAMHGcljPWdz\nZfDJlH0xAz/vuYDF3V8EUhvFiKQWbvPK+HXOwwrpdEba2I+fLYPX7bOhoDCitISOSDQp4tGeYxMM\nh4JZzUtendF6FYaqDEv6K8NXGSr46eu6YSn+yw+7Nxl3M81YbRU/H1zvarb5fHB9sjqo/ezOdYsf\nM0woIEnHrlukjXXIbNE6bn11l3/yQ2xf/HH+1HAJI8pKUoQf1AGsDC7U7FOs+Ewg8o1HkbOU7V5j\nPDhoRvBc2TxGEy+s9o4ZmbF3rlejrIDEbxrZmqNYFUG/HoqbYSLhcXzzr5fTHLOFmtrOkU1Q7f4B\ncDcRea3GfYu44/d4zcgr+NmhGWmfUwewUmh05V+MuLX+c2vzlwt+atHkUJfmqAkySjqTvWxFYGzg\nUE4tHu24PUW40Ryr5aPRH/DoZbu40NyZlmMQ7THJ1bcfQbXE2jLzWCGl2Z4M/ETxbG6+m8ia61N+\nj7eYu/hk6R9SPuM3DLVpWyszGzaqo1jxhYp/MeLHRJMrfmrRuI1xoccIhwm7dtcKSIYGK3jv8wqr\nhNSmKZtKb+ASnuLWx3ZlXX27Rdo4scQ612SsbLkETdtaqd66LFnv36Kk5whLRzyccxhqrjcnRVGz\nTzHix0STK376uk6uZ/MbB6l5bhnjzAEOmhGMks40kRdM0tTjhlV+wa33bgwI5NBMxelAniD7aQit\nYuEReDJ8Hu2R9H649gYsQLKOkLPukF2sc7XFZ+svu3ztbp7CPTqpIrKXp5fk5tzNdHPSKCHFDRX/\nYmTUBNc69HmXC55cH//PskOvmRf/mbgJNG1rZf4fTyLacyxyZmvpvLRmKdlW9xZuNXiCEm/XGDHl\nWZupgLsDuUK6WFDSyBNdH0kbb3f4AtQFn6aubCmUt9AZHsey6FXce2hGmlhXexSDy2Q6cnb3stPW\nHqGttIoJbuGpvfg9qqNYyRUV/2LErfVfocoFuxQr47EbAFi+tiotgmW0hwPXCsN0E/hsjJbDTDu6\n0tdY79j+/WlzhXg45vK1u7lp9XauGfkst5i7KOk5AkBFZA9LQnez5DOnweRLUj5ndSyzr65zLQmR\nMr/KMMverU8Le41QRrgXv8fe3JyU4Y3a/IuRRO18Rp0ESPynPR4/HzL4E9xWkW2myvNQhniET8ZG\n6i5ksu87iXn8EzYeecWd0VjSLj636xdJ4U/i4TtxC+3MtSSEnfmzJ7EueG5KP4FWU8ULZ36rV7/H\nQtQrUoYXuvIvViwTTaHJ4E+4ZuSzzO36RUr27bLueu4I3ekapmmNWVDSSE2WrlsWln1/dEWIxZee\nlrbadhLEvQZQQAxbS+dxa/fVniYjz05gHt9BJjNOrhyz/ZfykfbavOv4Z/MxKIoTzfBVUvne6e7+\nhPAYurs6U1bKnaaUhdG5TAv8L58Prk+5AXSaUh7s+Whaxq4XxkCrrZzD584+mW/XneEaW//vjc8n\newBn6+NrzdHtBuD52VEnQZY6/IoyWNEMX6V3eIV8QpqJxHKsLu7+IjdGr0vpk7swOpdZge2+hB/i\nZprarhVJkf7ty97N4O3N35d112c0KVlzrKkMp/XkdcsIHoytFjV+X+kLdOWvpLG5+W5Ofe5bjDJ/\nBYGu0CjKoh2uY2NGeO/RX7rue73sM55Zu06c9XcE+N5VU1ydrGUlgZQQzq2l8xgb8A4tBYEl7a71\neD5Z+geWjniYisje3mVKFzrT2oHbnP3UCFKGL7ryV3pF07ZWVm95k1JzNJ6ZC5RFOzwzc49UjKMm\nEVHi1HkvZ7BXsxU71ZVhz9j1Dkfs/q3dV6ev4O0kQifdnLa1/3gdFV9/GZa0x009uQp/oTOtHWin\nL6WvyEv8RWSMiKwTkVcSP0dnGHu8iLSIyA/zOafStyxfu5tvyM/Si6/Z/p8kFKbioqU8vfB8airD\naTcIN7NKl5Tx854L0kxEdpu8FaXiFaPuPE9zrJbFZl48ssg52GHGybeOfwp9kWntQOP3lb4i35X/\nQmCDMeZUYEPivRffAp7M83xKH3PtoR9lyM41nuGlbmLUHKtNCWVsiVXxjZ5/5o7Sa6ntWsF7j/4y\nxc4PqSGUucSo/6FiFmNubUUu/3HfhMC60ReZ1g6005fSV+Qb6nkZ8LHE63uB3wFfdw4SkWnAicD/\nBbLaopQBYkcjnytZ7915K0MUjFeSUXOsNq0aZ2UwvrrPZsd2S6zyInnuvgqBdaOvMq1tzJ89ifkP\nPk/UZitzZikrSm/Id+V/ojFmT+L1XuICn4KIBID/Ar6W57mUvmbDUs9/EAYyRsH4KZJm0RGJ+kqY\nsmz0fhDo/ygYP8XwCoHzbtxffTGVIU3Wlb+IrAfGuez6pv2NMcaIiJtf8DrgN8aYFsmS6y8i84B5\nACeffHK2qSmFJoO5QsJjMq6o3ZKMDh/t9iys5jdhqm5qDTeu3p51nEmcu18jYPwUw8uT5Wt3p5Wp\nsMpSa7SPkg9Zxd8Yc4HXPhH5i4iMN8bsEZHxwNsuw84BPiIi1wEjgVIROWSMSfMPGGNWAishHurp\n9yKUAuFlxkDgotuzftwp6F5hirmaLGo8TEpOBsQJ2sdmJnX4Kn1FvmafZuCaxOtrgEedA4wxnzXG\nnGyMOYW46ec+N+FXBgGu9foFpn+xVwJXqHo4fk1KQ9EJqg5fpa/I1+HbADSKyJeAPwP1ACIyHfiy\nMWZunsdX+pM+MGMUoh6O06RUWRHi0JHuFCfoUC1iVuhqoopioRm+SlHiVvNnqNrAh9O1KvnjN8NX\nxV9RFGUIoeUdFEVRFE9U/BVFUYYhKv6KoijDEBV/RVGUYYiKv6IoyjBExV9RFGUYouKvKIoyDFHx\nVxRFGYao+CuKogxDVPwVRVGGISr+iqIowxAVf0VRlGFIviWdlWGOVpxUlOJExV/pNc5OXa3tEW5e\nsxNAbwCKMshRs4/Sa5av3Z3SZAQgEu1h+drdAzQjRVH8ouKv9BrtL6soxYuKv9JrtL+sohQvKv5K\nr3FrrK79ZRWlOFCHr9JrnI3VNdpHUYqHvMRfRMYAq4FTgDeAemPMQZdxJwOrgJMAA1xsjHkjn3Mr\ng4O6qTUq9opShORr9lkIbDDGnApsSLx34z5guTHmA8AM4O08z6soiqLkQb7ifxlwb+L1vUCdc4CI\nfBAoMcasAzDGHDLGdOZ5XkVRFCUP8hX/E40xexKv9wInuoz5O6BdRNaIyDYRWS4iQZdxiqIoSj+R\n1eYvIuuBcS67vml/Y4wxImI8zvERYCrwJnEfwT8BP3E51zxgHsDJJ5+cbWqKoihKL8kq/saYC7z2\nichfRGS8MWaPiIzH3ZbfAmw3xrye+EwTcDYu4m+MWQmsBJg+fbrbjURRFEUpAPmafZqBaxKvrwEe\ndRmzGagUkRMS788HXszzvIqiKEoe5Cv+DcCFIvIKcEHiPSIyXURWARhjeoCvARtEZCcgwI/zPK+i\nKIqSB3nF+RtjDgCzXLZvAeba3q8DJudzLkVRFKVwaHkHRVGUYYiKv6IoyjBExV9RFGUYouKvKIoy\nDFHxVxRFGYZoSecCoY3MFUUpJlT8C4A2MlcUpdhQs08B0EbmiqIUGyr+BUAbmSuKUmyo+BcAbWSu\nKEqxoeJfALSRuaIoxYY6fAuANjJXFKXYUPEvENrIXFGUYkLNPoqiKMMQFX9FUZRhiIq/oijKMETF\nX1EUZRii4q8oijIMUfFXFEUZhqj4K4qiDEPyEn8RGSMi60TklcTP0R7jlonILhF5SURWiIjkc15F\nURQlP/Jd+S8ENhhjTgU2JN6nICIfBmYCk4HTgbOAc/M8r6IoipIH+Yr/ZcC9idf3AnUuYwxQDpQC\nZUAI+Eue51UURVHyIF/xP9EYsyfxei9wonOAMeb/Ab8F9iT+W2uMecntYCIyT0S2iMiWffv25Tk1\nRVEUxYustX1EZD0wzmXXN+1vjDFGRIzL5/8W+AAwIbFpnYh8xBjzlHOsMWYlsBJg+vTpacdSFEVR\nCkNW8TfGXOC1T0T+IiLjjTF7RGQ88LbLsH8EnjHGHEp85gngHCBN/BVFUZT+IV+zTzNwTeL1NcCj\nLmPeBM4VkRIRCRF39rqafRRFUZT+IV/xbwAuFJFXgAsS7xGR6SKyKjHmIeA1YCfwPPC8MeaxPM+r\nKIqi5EFe9fyNMQeAWS7btwBzE697gGvzOY+iKIpSWDTDV1EUZRii4q8oijIMUfFXFEUZhqj4K4qi\nDENU/BVFUYYhKv6KoijDkLxCPRVvmra1snztbtraI1RXhpk/exJ1U2sGelqKoiiAin+f0LStlZvX\n7CQS7QGgtT3CzWt2AugNQFGUQYGaffqA5Wt3J4XfIhLtYfna3QM0I0VRlFR05V9gmra10toecd3X\n5rGdHY2wYSl0tMCoCTBrEU09M/vNbKQmKkUZfqj428hFBN3GAknzjp05gU0sKGmkOnAAvpcq7tPf\nXUdD6U8IczQ+uOMtuh/9VzZF59La9WGgAGYjl5sLk+uT16EmKkUZfogxg7Ns/vTp082WLVv67XxO\nEQQIh4LcdvkZaSK4ufluqrcuYzz7aTNVLOuuZ13wXMpDAQ52RlPGzglsoiG0igrpSm7rkjK+0fPP\nPNT1YTaV3sCEwP60+bTEqqjtWpGyraYyzNMLz8/twnY0wmM3QNT21BEKw6UrYHI9Mxs2uj6p9Opc\niqIMOCKy1RgzPdu4IWnzb9rWysyGjUxc+DgzGzbStK0162f82uk3N9/NaVtvoUb2ExCYENhPQ2gV\nF/b8Pk34ARaUNKYIP0CpOcqNPABAtaQLf3z7gbRtnmajTGxYmir8EH+/YWnGY/bqXIqiFA1DTvyt\nFXxrewTDMTNGthuAHxFs2tZK9dZlaWJeIV0sKGl0/Xw2cW8zVe7nNWPTP1MZdh2bkY6WjNu9jtmr\ncymKUjQMOfHvbaSNHxFcvnY34/EW88pwiHAomLI9m7gv666n05Sm7Os0pSzrrk/ZFg4Fk36FnBg1\nIeP2+bMnpc251+dSFKVoGHLi31szhh8RbGuPeIr5HsayZM5p3Hb5GdRUhhHidvMfBT7jKu7LE+Le\nHKtlYXQuLbEqYkZoiVWxMDqX5lhtcnxNZdjV9+CLWYviNn47oXB8O3GnrnPOvT6XoihFw5CL9qmu\nDLs6MLOZMSyxyxTtU10ZZtm79WkO3E5TStu0Bcmx9s80bZvEokdi3GgeoFoO0GbGcgef4rgZn6Lm\n5X20tUfYevyFXNp1nqvPIG/HayKqxyvax5qvir2iDC+GnPjPnz3JNWrHjxkjmwjGj90F0bgjt1oO\nsIextE1bwFlz3JuVxY93HVetnZUxhNQr2qgg5pfJ9SliryiKMuTE388KPv9jl/KR9lrfx/azsu7L\neSuKojjJK85fRK4ElgAfAGYkeve6jfsE8H0gCKwyxjRkO3Z/x/kriqIMBforzv8F4HLgyQwTCQI/\nAi4CPgh8WkQ+mOd5FUVRlDzIy+xjjHkJQEQyDZsBvGqMeT0x9gHgMuDFfM6tKIqi9J7+CPWsAd6y\nvW9JbFMURVEGiKwrfxFZD4xz2fVNY8yjhZyMiMwD5gGcfPLJhTy0oiiKYiOr+BtjLsjzHK3ASbb3\nExLb3M61ElgJcYdvnudVFEVRPOgPs89m4FQRmSgipcCngOZ+OK+iKIriQV7iLyL/KCItwDnA4yKy\nNrG9WkR+A2CM6QauB9YCLwGNxphd+U1bURRFyYdBW89fRPYBf/YxtAo8qq0VL3pNxcFQvCYYmtc1\nnK7pPcaYE7J9eNCKv19EZIufhIZiQq+pOBiK1wRD87r0mtIZclU9FUVRlOyo+CuKogxDhoL4rxzo\nCfQBek3FwVC8Jhia16XX5KDobf6KoihK7gyFlb+iKIqSI0Un/iJypYjsEpGYiHh6ukXkDRHZKSLb\nRWRQ14bO4Zo+ISK7ReRVEVnYn3PMFREZIyLrROSVxM/RHuN6Er+j7SIyKJP/sn3vIlImIqsT+/8o\nIqf0/yxzw8c1/ZOI7LP9buYOxDxzQUTuEZG3ReQFj/0iIisS17xDRM7s7znmio9r+piIdNh+T4t8\nH9wYU1T/Ee8dMAn4HTA9w7g3gKqBnm+hrol4L4TXgPcCpcDzwAcHeu4ZrmkZsDDxeiFwu8e4QwM9\n1yzXkfV7B64D7kq8/hSweqDnXYBr+ifghwM91xyv66PAmcALHvsvBp4ABDgb+ONAz7kA1/Qx4Ne9\nOXbRrfyNMS8ZY3YP9DwKic9rSpbGNsZ0AVZp7MHKZcC9idf3AnUDOJd88PO926/1IWCWZKlzPsAU\n278lXxhjngTeyTDkMtfkNxQAAAJ/SURBVOA+E+cZoFJExvfP7HqHj2vqNUUn/jlggP8Rka2JaqHF\nTrGVxj7RGLMn8XovcKLHuHIR2SIiz4jIYLxB+Pnek2NMvJxJBzC2X2bXO/z+W7oiYR55SEROctlf\nbBTb35BfzhGR50XkCRE5ze+HBmUP3wKVka41xrSKyN8A60Tk5cRddEDoz9LY/UWma7K/McYYEfEK\nK3tP4vf0XmCjiOw0xrxW6LkqOfMYcL8x5qiIXEv8yeb8AZ6Tks5zxP+GDonIxUATcKqfDw5K8Tf5\nl5HGGNOa+Pm2iDxC/FF3wMS/ANfkuzR2f5HpmkTkLyIy3hizJ/Fo/bbHMazf0+si8jtgKnF79GDB\nz/dujWkRkRJgFHCgf6bXK7JekzHGPv9VxH04xc6g+xvKF2PMu7bXvxGRO0WkyhiTtY7RkDT7iMgI\nETnOeg18nHi/4WKm2EpjNwPXJF5fA6Q93YjIaBEpS7yuAmYy+Np7+vne7df6SWCjSXjjBilZr8lh\nC59DvCJvsdMMXJ2I+jkb6LCZJosSERln+ZdEZAZxTfe38Bhob3YvvN//SNxWdxT4C7A2sb0a+E3i\n9XuJRzA8D+wibloZ8Lnnc02J9xcD/0t8ZTzYr2kssAF4BVgPjElsnw6sSrz+MLAz8XvaCXxpoOft\ncS1p3zuwFJiTeF0OPAi8CjwLvHeg51yAa7ot8bfzPPBb4P0DPWcf13Q/sAeIJv6evgR8GfhyYr8A\nP0pc804yRAsOlv98XNP1tt/TM8CH/R5bM3wVRVGGIUPS7KMoiqJkRsVfURRlGKLiryiKMgxR8VcU\nRRmGqPgriqIMQ1T8FUVRhiEq/oqiKMMQFX9FUZRhyP8H7j+Bxx0MFoAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f6925d6bd90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred_train = model.predict(sme.transform(X_train))\n",
    "y_pred_test = model.predict(sme.transform(X_test))\n",
    "plt.scatter(y_train, y_pred_train, label=\"Train\")\n",
    "plt.scatter(y_test, y_pred_test, label=\"Test\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not the best model until now. However, prolonged training with lowering of the learning rate towards the end will improve the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-0.38300244]]\n"
     ]
    }
   ],
   "source": [
    "#The Enumerator can be used in sampling\n",
    "i = 1\n",
    "\n",
    "y_true = y_test[i]\n",
    "y_pred = model.predict(sme.transform(X_test.iloc[i:i+1]))\n",
    "print y_true - y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0616476\n",
      "[-0.10707214]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD8CAYAAACRkhiPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEytJREFUeJzt3X+QXWd93/H3p/6VqfEUGy3C2JZl\nGtczJhNMuhVJIYnBRraFg2nrtvKkREmcUUJhJkzptKJMcceZzjjtJExbZ6KqWLFJiaEpMWhigREG\n1zDDL8mVbdngSDjOWEJYwjb+ESgZkW//2KP2sr6r3b3n7u7Fz/s1c+ee85znnOerc1f72XPuueem\nqpAktelvrHQBkqSVYwhIUsMMAUlqmCEgSQ0zBCSpYYaAJDXMEJCkhhkCktQwQ0CSGnbyShcwzKpV\nq2rt2rUrXYYk/cjYs2fPt6tqarHrTWQIrF27lt27d690GZL0IyPJX4yynqeDJKlhhoAkNcwQkKSG\nGQKS1DBDQJIaNm8IJDkvyeeSPJzkoSS/2bWflWRXkv3d85lzrL+p67M/yaZx/wMkSaNbyJHAMeA9\nVXUx8NPAO5NcDGwB7q6qC4G7u/kfkuQs4AbgdcA64Ia5wkKStPzmDYGqOlxV93XTzwFfA84BrgFu\n67rdBrxtyOpXALuq6qmqehrYBVw5jsIlSf0t6j2BJGuB1wJfBlZX1eFu0beA1UNWOQd4fGD+YNcm\nSZoAC/7EcJKXAB8D3l1Vzyb5f8uqqpL0+sb6JJuBzQBr1qzps6nmrN1y54qM+9hNb1mRcSWNz4KO\nBJKcwkwAfLiq/qRrfiLJ2d3ys4EjQ1Y9BJw3MH9u1/YCVbWtqqaranpqatG3v5AkjWAhVwcFuAX4\nWlX97sCiHcDxq302AZ8YsvpdwPokZ3ZvCK/v2iRJE2AhRwKvB94OvCnJ3u6xAbgJeHOS/cDl3TxJ\nppN8EKCqngJ+C/hq97ixa5MkTYB53xOoqi8AmWPxZUP67wZ+bWB+O7B91AIlSUvHTwxLUsMMAUlq\nmCEgSQ0zBCSpYYaAJDXMEJCkhhkCktQwQ0CSGmYISFLDDAFJapghIEkNMwQkqWGGgCQ1zBCQpIYZ\nApLUMENAkhpmCEhSw+b9ZrEk24GrgSNV9RNd20eBi7ouLwW+U1WXDFn3MeA54AfAsaqaHlPdkqQx\nmDcEgFuBm4EPHW+oqn96fDrJ7wDPnGD9N1bVt0ctUJK0dBbyHcP3Jlk7bFmSAP8EeNN4y5IkLYe+\n7wn8LPBEVe2fY3kBn06yJ8nmnmNJksZsIaeDTuQ64PYTLH9DVR1K8nJgV5KvV9W9wzp2IbEZYM2a\nNT3LkiQtxMhHAklOBv4h8NG5+lTVoe75CHAHsO4EfbdV1XRVTU9NTY1aliRpEfqcDroc+HpVHRy2\nMMnpSc44Pg2sB/b1GE+SNGbzhkCS24EvAhclOZjk+m7RRmadCkryyiQ7u9nVwBeS3A98Bbizqj41\nvtIlSX0t5Oqg6+Zo/+Uhbd8ENnTTjwKv6VmfJGkJ+YlhSWqYISBJDTMEJKlhhoAkNcwQkKSGGQKS\n1DBDQJIaZghIUsMMAUlqmCEgSQ0zBCSpYYaAJDXMEJCkhhkCktQwQ0CSGmYISFLDDAFJathCvl5y\ne5IjSfYNtP27JIeS7O0eG+ZY98okjyQ5kGTLOAuXJPW3kCOBW4Erh7R/oKou6R47Zy9MchLwe8BV\nwMXAdUku7lOsJGm85g2BqroXeGqEba8DDlTVo1X1V8BHgGtG2I4kaYnM+0XzJ/CuJL8E7AbeU1VP\nz1p+DvD4wPxB4HVzbSzJZmAzwJo1a3qUpeWydsudKzb2Yze9ZcXGll5MRn1j+PeBvw1cAhwGfqdv\nIVW1raqmq2p6amqq7+YkSQswUghU1RNV9YOq+mvgvzFz6me2Q8B5A/Pndm2SpAkxUggkOXtg9h8A\n+4Z0+ypwYZILkpwKbAR2jDKeJGlpzPueQJLbgUuBVUkOAjcAlya5BCjgMeDXu76vBD5YVRuq6liS\ndwF3AScB26vqoSX5V0iSRjJvCFTVdUOab5mj7zeBDQPzO4EXXD4qSZoMfmJYkhpmCEhSwwwBSWqY\nISBJDTMEJKlhhoAkNcwQkKSGGQKS1DBDQJIaZghIUsMMAUlqmCEgSQ0zBCSpYYaAJDXMEJCkhhkC\nktQwQ0CSGjZvCCTZnuRIkn0Dbf8xydeTPJDkjiQvnWPdx5I8mGRvkt3jLFyS1N9CjgRuBa6c1bYL\n+Imq+kngz4D3nmD9N1bVJVU1PVqJkqSlMm8IVNW9wFOz2j5dVce62S8B5y5BbZKkJTaO9wR+Ffjk\nHMsK+HSSPUk2n2gjSTYn2Z1k99GjR8dQliRpPr1CIMn7gGPAh+fo8oaq+ingKuCdSX5urm1V1baq\nmq6q6ampqT5lSZIWaOQQSPLLwNXAL1ZVDetTVYe65yPAHcC6UceTJI3fSCGQ5ErgXwFvrarvztHn\n9CRnHJ8G1gP7hvWVJK2MhVwiejvwReCiJAeTXA/cDJwB7Oou/9za9X1lkp3dqquBLyS5H/gKcGdV\nfWpJ/hWSpJGcPF+HqrpuSPMtc/T9JrChm34UeE2v6iRJS8pPDEtSwwwBSWqYISBJDTMEJKlhhoAk\nNcwQkKSGGQKS1DBDQJIaZghIUsMMAUlq2Ly3jdDCrN1y50qXIEmL5pGAJDXMEJCkhhkCktQwQ0CS\nGmYISFLDDAFJatiCQiDJ9iRHkuwbaDsrya4k+7vnM+dYd1PXZ3+STeMqXJLU30KPBG4FrpzVtgW4\nu6ouBO7u5n9IkrOAG4DXAeuAG+YKC0nS8ltQCFTVvcBTs5qvAW7rpm8D3jZk1SuAXVX1VFU9Dezi\nhWEiSVohfd4TWF1Vh7vpbwGrh/Q5B3h8YP5g1/YCSTYn2Z1k99GjR3uUJUlaqLG8MVxVBVTPbWyr\nqumqmp6amhpHWZKkefQJgSeSnA3QPR8Z0ucQcN7A/LldmyRpAvQJgR3A8at9NgGfGNLnLmB9kjO7\nN4TXd22SpAmw0EtEbwe+CFyU5GCS64GbgDcn2Q9c3s2TZDrJBwGq6ingt4Cvdo8buzZJ0gRY0K2k\nq+q6ORZdNqTvbuDXBua3A9tHqk6StKT8xLAkNcwQkKSGGQKS1DBDQJIaZghIUsMMAUlqmCEgSQ0z\nBCSpYYaAJDXMEJCkhhkCktQwQ0CSGmYISFLDDAFJapghIEkNMwQkqWEjh0CSi5LsHXg8m+Tds/pc\nmuSZgT7v71+yJGlcFvTNYsNU1SPAJQBJTmLmC+TvGNL181V19ajjSJKWzrhOB10GfKOq/mJM25Mk\nLYNxhcBG4PY5lv1MkvuTfDLJq8c0niRpDHqHQJJTgbcCfzxk8X3A+VX1GuC/AB8/wXY2J9mdZPfR\no0f7liVJWoBxHAlcBdxXVU/MXlBVz1bV8930TuCUJKuGbaSqtlXVdFVNT01NjaEsSdJ8xhEC1zHH\nqaAkr0iSbnpdN96TYxhTkjQGI18dBJDkdODNwK8PtP0GQFVtBa4F3pHkGPA9YGNVVZ8xJUnj0ysE\nquovgZfNats6MH0zcHOfMSRJS8dPDEtSwwwBSWqYISBJDTMEJKlhhoAkNcwQkKSGGQKS1DBDQJIa\nZghIUsMMAUlqmCEgSQ0zBCSpYYaAJDXMEJCkhhkCktQwQ0CSGmYISFLDeodAkseSPJhkb5LdQ5Yn\nyX9OciDJA0l+qu+YkqTx6PX1kgPeWFXfnmPZVcCF3eN1wO93z5KkFbYcp4OuAT5UM74EvDTJ2csw\nriRpHuMIgQI+nWRPks1Dlp8DPD4wf7BrkyStsHGcDnpDVR1K8nJgV5KvV9W9i91IFyCbAdasWTOG\nsiRJ8+l9JFBVh7rnI8AdwLpZXQ4B5w3Mn9u1zd7OtqqarqrpqampvmVJkhagVwgkOT3JGcengfXA\nvlnddgC/1F0l9NPAM1V1uM+4kqTx6Hs6aDVwR5Lj2/qjqvpUkt8AqKqtwE5gA3AA+C7wKz3HlCSN\nSa8QqKpHgdcMad86MF3AO/uMI0laGn5iWJIaZghIUsMMAUlqmCEgSQ0zBCSpYYaAJDXMEJCkhhkC\nktQwQ0CSGmYISFLDDAFJapghIEkNMwQkqWGGgCQ1zBCQpIYZApLUMENAkho2cggkOS/J55I8nOSh\nJL85pM+lSZ5Jsrd7vL9fuZKkcerz9ZLHgPdU1X3dl83vSbKrqh6e1e/zVXV1j3EkSUtk5COBqjpc\nVfd1088BXwPOGVdhkqSlN5b3BJKsBV4LfHnI4p9Jcn+STyZ59TjGkySNR5/TQQAkeQnwMeDdVfXs\nrMX3AedX1fNJNgAfBy6cYzubgc0Aa9as6VuWJGkBeh0JJDmFmQD4cFX9yezlVfVsVT3fTe8ETkmy\nati2qmpbVU1X1fTU1FSfsiRJC9Tn6qAAtwBfq6rfnaPPK7p+JFnXjffkqGNKksarz+mg1wNvBx5M\nsrdr+zfAGoCq2gpcC7wjyTHge8DGqqoeY0qSxmjkEKiqLwCZp8/NwM2jjiFJWlp+YliSGmYISFLD\nDAFJapghIEkNMwQkqWGGgCQ1zBCQpIYZApLUMENAkhpmCEhSw3rfSnrSrN1y50qXoGWwUq/zYze9\nZUXGXUkt7uuW/s0eCUhSwwwBSWqYISBJDTMEJKlhhoAkNcwQkKSG9f2i+SuTPJLkQJItQ5afluSj\n3fIvJ1nbZzxJ0nj1+aL5k4DfA64CLgauS3LxrG7XA09X1Y8DHwB+e9TxJEnj1+dIYB1woKoeraq/\nAj4CXDOrzzXAbd30/wQuS3LC7yWWJC2fPiFwDvD4wPzBrm1on6o6BjwDvKzHmJKkMZqY20Yk2Qxs\n7mafT/LIEgyzCvj2Emx3HCa1tkmtC1agtizshKb7bPFeUNcC9/VyWLZ9tsh/8+y6zh9lzD4hcAg4\nb2D+3K5tWJ+DSU4G/hbw5LCNVdU2YFuPeuaVZHdVTS/lGKOa1NomtS6Y3NomtS6Y3NomtS6Y3NrG\nVVef00FfBS5MckGSU4GNwI5ZfXYAm7rpa4HPVlX1GFOSNEYjHwlU1bEk7wLuAk4CtlfVQ0luBHZX\n1Q7gFuAPkxwAnmImKCRJE6LXewJVtRPYOavt/QPT/wf4x33GGLMlPd3U06TWNql1weTWNql1weTW\nNql1weTWNpa64tkZSWqXt42QpIa96EIgyVlJdiXZ3z2fOUe/TyX5TpI/ndV+a5I/T7K3e1wyQbVd\n0N1+40B3O45Tl7muTV2f/Uk2DbTf090+5Pg+e3nPeka+HUmS93btjyS5ok8d46wtydok3xvYR1uX\nua6fS3JfkmNJrp21bOjrOiG1/WBgn82+8GSp6/oXSR5O8kCSu5OcP7BspffZiWpb3D6rqhfVA/gP\nwJZuegvw23P0uwz4BeBPZ7XfClw7obX9D2BjN70VeMdy1QWcBTzaPZ/ZTZ/ZLbsHmB5TLScB3wBe\nBZwK3A9cPKvPPwe2dtMbgY920xd3/U8DLui2c9IYX78+ta0F9i3Rz9VC6loL/CTwocGf7xO9ritd\nW7fs+RXcZ28E/mY3/Y6B13IS9tnQ2kbZZy+6IwF++FYVtwFvG9apqu4Gnluuojoj15YkwJuYuf3G\nCddforquAHZV1VNV9TSwC7hyTOMP6nM7kmuAj1TV96vqz4ED3fYmobalNG9dVfVYVT0A/PWsdZf6\nde1T21JaSF2fq6rvdrNfYuazUDAZ+2yu2hbtxRgCq6vqcDf9LWD1CNv4991h1geSnDYhtb0M+E7N\n3H4Dht+mYynrmu82IX/QHX7+256/9PrcjmQh6/bR91YpFyT530n+V5KfXea6lmLd5dj+jyXZneRL\nScb1R88odV0PfHLEdZezNljkPpuY20YsRpLPAK8Ysuh9gzNVVUkWe/nTe5n5RXgqM5dg/Wvgxgmp\nbWRLXNcvVtWhJGcAHwPezsyhvf6/w8Caqnoyyd8FPp7k1VX17EoXNuHO7362XgV8NsmDVfWN5Swg\nyT8DpoGfX85xF2KO2ha1z34kQ6CqLp9rWZInkpxdVYeTnA0cWeS2j/9F/P0kfwD8ywmp7UngpUlO\n7v7CHHabjqWs6xBw6cD8ucy8F0BVHeqen0vyR8wczo4aAn1uR7KQdfsYubaaOVn7fYCq2pPkG8Df\nAXYvU10nWvfSWeveM4aaBrc/8msy8LP1aJJ7gNcyc758WepKcjkzfyj9fFV9f2DdS2ete88YahpH\nbYveZy/G00GDt6rYBHxiMSt3vwSPn4N/G7BvEmrrfol8jpnbbyx6/THUdRewPsmZmbl6aD1wV5KT\nk6wCSHIKcDX99lmf25HsADZm5gqdC4ALga/0qGVstSWZysx3cND9hXYhM28oLlddcxn6uo6prl61\ndTWd1k2vAl4PPLxcdSV5LfBfgbdW1eAfRiu+z+aqbaR9Nq53tCflwcz517uB/cBngLO69mnggwP9\nPg8cBb7HzDm3K7r2zwIPMvOL7L8DL5mg2l7FzC+1A8AfA6ctc12/2o19APiVru10YA/wAPAQ8J/o\neUUOsAH4M2b+enlf13Zj9wMP8GPdv/9Atz9eNbDu+7r1HgGuWoKfr5FqA/5Rt3/2AvcBv7DMdf29\n7mfpL5k5anroRK/rJNQG/P3u/+L93fP1y1zXZ4AnutdsL7BjgvbZ0NpG2Wd+YliSGvZiPB0kSVog\nQ0CSGmYISFLDDAFJapghIEkNMwQkqWGGgCQ1zBCQpIb9XzgIarIc42zVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f691ced8390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Enumeration of the SMILES before sampling stabilises the result\n",
    "smiles_repeat = np.array([X_test.iloc[i:i+1].values[0]]*50)\n",
    "y_pred = model.predict(sme.transform(smiles_repeat))\n",
    "print y_pred.std()\n",
    "print y_true - np.median(y_pred)\n",
    "_ = plt.hist(y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bibliography\n",
    "\n",
    "Please cite: [SMILES enumeration as Data Augmentation for Network Modeling of Molecules](https://arxiv.org/abs/1703.07076)\n",
    "\n",
    "```bibtex\n",
    "@article{DBLP:journals/corr/Bjerrum17,\n",
    "  author    = {Esben Jannik Bjerrum},\n",
    "  title     = {{SMILES} Enumeration as Data Augmentation for Neural Network Modeling\n",
    "               of Molecules},\n",
    "  journal   = {CoRR},\n",
    "  volume    = {abs/1703.07076},\n",
    "  year      = {2017},\n",
    "  url       = {http://arxiv.org/abs/1703.07076},\n",
    "  timestamp = {Wed, 07 Jun 2017 14:40:38 +0200},\n",
    "  biburl    = {http://dblp.uni-trier.de/rec/bib/journals/corr/Bjerrum17},\n",
    "  bibsource = {dblp computer science bibliography, http://dblp.org}\n",
    "}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you find it useful, feel welcome to leave a comment on the [blog.](https://www.wildcardconsulting.dk/useful-information/smiles-enumeration-as-data-augmentation-for-molecular-neural-networks/) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Keras2.1.1-Tensorflow",
   "language": "python",
   "name": "keras2.1.1-tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
